Generating adaptive textual explanations of output predicted by trained artificial-intelligence processes

ABSTRACT

The disclosed embodiments include computer-implemented processes that generate adaptive textual explanations of output using trained artificial intelligence processes. For example, an apparatus may generate an input dataset based on elements of first interaction data associated with a first temporal interval, and based on an application of a trained artificial intelligence process to the input dataset, generate output data representative of a predicted likelihood of an occurrence of an event during a second temporal interval. Further, and based on an application of a trained explainability process to the input dataset, the apparatus may generate an element of textual content that characterizes an outcome associated with the predicted likelihood of the occurrence of the event, where the element of textual content is associated with a feature value of the input dataset. The apparatus may also transmit a portion of the output data and the element of textual content to a computing system.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of priority to U.S. ProvisionalPatent Application No. 63/252,496, filed on Oct. 5, 2021, the entiredisclosure of which is expressly incorporated herein by reference to itsentirety.

TECHNICAL FIELD

The disclosed embodiments generally relate to computer-implementedsystems and processes that generate adaptive textual explanations ofoutput predicted by trained artificial intelligence processes.

BACKGROUND

Today, many financial institutions extend credit in the form ofcredit-card accounts, personal loans, and other unsecuredlines-of-credit to their customers in accordance with certain terms andconditions, such as a repayment schedule or corresponding interest rate.The terms and conditions associated with the extended credit may beestablished initially by the financial institutions prior to issuing thecredit-card accounts, personal loans, and unsecured lines-of-credit tocorresponding ones of the customers and further, the financialinstitutions may elect to modify one or more of the terms and conditionsof the extended credit based on an evolution in the relationshipsbetween the financial institutions and the customers, and based on thecustomer's use, or misuse, of various financial or credit instrumentsissued by these financial institutions.

SUMMARY

In some examples, an apparatus includes a memory storing instructions, acommunications interface, and at least one processor coupled to thememory and the communications interface. The at least one processor isconfigured to execute the instructions to generate an input datasetbased on elements of first interaction data associated with a firsttemporal interval. Based on an application of a trained artificialintelligence process to the input dataset, the at least one processor isfurther configured to execute the instructions to generate output datarepresentative of a predicted likelihood of an occurrence of an eventduring a second temporal interval. Based on an application of a trainedexplainability process to the input dataset, the at least one processoris further configured to execute the instructions to generate a firstelement of textual content that characterizes an outcome associated withthe predicted likelihood of the occurrence of the event. The firstelement of textual content is associated with a feature value of theinput dataset. The at least one processor is further configured toexecute the instructions to transmit a portion of the output data andthe first element of textual content to a computing system via thecommunications interface. The computing system is configured to generateor modify second interaction data based on the portion of the outputdata, and to provision notification data comprising the first element oftextual content to a device associated with the first interaction data.

In other examples, a computer-implemented method includes generating,using at least one processor, an input dataset based on elements offirst interaction data associated with a first temporal interval. Thecomputer-implemented method also includes, using the at least oneprocessor, and based on an application of a trained artificialintelligence process to the input dataset, generating output datarepresentative of a predicted likelihood of an occurrence of an eventduring a second temporal interval. Further, the computer-implementedmethod includes, using the at least one processor, and based on anapplication of a trained explainability process to the input dataset,generating a first element of textual content that characterizes anoutcome associated with the predicted likelihood of the occurrence ofthe event. The first element of textual content is associated with afeature value of the input dataset. The method also includes, using theat least one processor, transmitting a portion of the output data andthe first element of textual content to a computing system. Thecomputing system is configured to generate or modify second interactiondata based on the portion of the output data, and to provisionnotification data comprising the first element of textual content to adevice associated with the first interaction data.

Further, in some examples, a tangible, non-transitory computer-readablemedium stores instructions that, when executed by at least oneprocessor, cause the at least one processor to perform a method thatincludes generating an input dataset based on elements of firstinteraction data associated with a first temporal interval. The methodalso includes, based on an application of a trained artificialintelligence process to the input dataset, generating output datarepresentative of a predicted likelihood of an occurrence of an eventduring a second temporal interval. Further, the method includes, andbased on an application of a trained explainability process to the inputdataset, generating a first element of textual content thatcharacterizes an outcome associated with the predicted likelihood of theoccurrence of the event. The first element of textual content isassociated with a feature value of the input dataset. The method alsoincludes transmitting a portion of the output data and the first elementof textual content to a computing system. The computing system isconfigured to generate or modify second interaction data based on theportion of the output data, and to provision notification datacomprising the first element of textual content to a device associatedwith the first interaction data.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the invention, as claimed. Further, theaccompanying drawings, which are incorporated in and constitute a partof this specification, illustrate aspects of the present disclosure andtogether with the description, serve to explain principles of thedisclosed exemplary embodiments, as set forth in the accompanyingclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are block diagrams illustrating portions of an exemplarycomputing environment, in accordance with some exemplary embodiments.

FIGS. 1C and 1D are diagrams of exemplary timelines for training amachine-learning or artificial intelligence process, in accordance withsome exemplary embodiments.

FIG. 2A illustrates an exemplary Shapley scatter plot, in accordancewith some exemplary embodiments.

FIGS. 2B, 3A, 3B, and 3C are block diagrams illustrating additionalportions of the exemplary computing environment, in accordance with someexemplary embodiments.

FIG. 4 is a flowchart of an exemplary processes for training amachine-learning or artificial intelligence process, in accordance withsome embodiments.

FIG. 5 is a flowchart of exemplary processes for training anexplainability process, in accordance with some embodiments.

FIG. 6A is flowchart of an exemplary process for generating textual datacharacterizing a predicted output of a trained machine-learning orartificial-intelligence process, in accordance with some embodiments.

FIG. 6B is flowchart of an exemplary process for applying one or moreexplainability processes to an input dataset associated with a trainedmachine-learning or artificial-intelligence process, in accordance withsome embodiments.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

Modern financial institutions offer a variety of financial products orservices to their customers, both through in-person branch banking andthrough various digital channels, and decisions related to theprovisioning of a particular financial product or financial service to acorresponding customer are often informed by the customer's relationshipwith the financial institution and the customer's use, or misuse, ofother financial products or services. For example, one or more computingsystems of a financial institution (e.g., an FI computing system, asdescribed herein) may obtain, generate, and maintain elements ofcustomer profile data identifying the customer and characterizing thecustomer's relationship with the financial institution, elements ofaccount data identifying and characterizing one or more financialproducts issued to the customer by the financial institution, elementsof transaction data identifying and characterizing one or moretransactions involving these issued financial products, or elements ofreporting data, such as credit-bureau data associated with theparticular customer. The elements of customer profile data, accountdata, transaction data, and/or reporting data may establish collectivelya time-evolving risk profile for the customer, and the financialinstitution may base not only a decision to provision the particularfinancial product or service to the corresponding customer, but also adetermination of one or more initial terms and conditions of theprovisioned financial product or service, on the established riskprofile.

Further, and as described herein, the time-evolving risk profile of thecustomer may also inform decisions by the financial institution thatimpact the provisioned product or service, such as, but not limited to,a decision by the financial institution to modify one or more of theterms and conditions imposed initially on the provisioned product orservice (e.g., an increase or decrease in a credit limit, a change in arepayment schedule, etc.), or a decision by the financial institution toauthorize a transaction involving the provisioned product or service.Further, the time-evolving risk profile of the customer, either alone orin conjunction with additional elements of the customer profile data,account data, transaction data, and/or reporting data that characterizea use, or misuse, of the provisioned product or service, may also informdecisions by the financial institution regarding a suspension or closureof the provisioned product or service, or a subsequent re-issuance ofthat product or service, and additionally, or alternatively, may alsoinform one or more collection activities or strategies associated withthe customer or the provisioned product or service (e.g., aprioritization of collection activities, etc.).

In some instances, to further characterize the time-evolving riskprofile of the customer, and to further inform the decisions by thefinancial institution regarding a particular financial product orservice provisioned, or available for provisioning, to the customer, amachine-learning or artificial-intelligence process may be trained topredict a likelihood of an occurrence of one or more events associatedwith, or involving, a customer of the financial institution and acorresponding financial product or service during a future temporalinterval using training data associated with a first prior temporalinterval, and using validation data associated with a second, anddistinct, prior temporal interval. The machine-learning orartificial-intelligence process may include an ensemble or decision-treeprocess, such as a gradient-boosted decision-tree process (e.g., XGBoostprocess), and the trained machine-learning or artificial-intelligenceprocess (e.g., the trained gradient-boosted, decision-tree processdescribed herein) may further ingest input datasets associated with oneor more customers of the financial institution, and based on anapplication of the trained gradient-boosted, decision-tree process tothe input datasets, the one or more FI computing systems may generateelements of output data indicative of a likelihood of an occurrence ofone or more events involving corresponding ones of the customers and thecorresponding financial product or service during a future temporalinterval disposed subsequent to a prediction date.

By way of example, the corresponding financial product or service mayinclude, but is not limited to, a credit product, such as a secured orunsecured credit-card account held by a corresponding customer of thefinancial institution, such as, but not limited to, an individual orpersonal-banking customer or a small-business banking customer. Further,and through an implementation of the exemplary processes describedherein, the one or more FI computing systems (e.g., which maycollectively establish a distributed computing cluster associated withthe financial institution) may adaptively, and successively, train andvalidate the machine-learning or artificial-intelligence process topredict an occurrence of a default event involving credit card accountheld by a customer of the financial during a future, twelve-monthinterval using respective elements of the training and validation data.

In some instances, the training and validation data associated with theprediction of the occurrence of the default event may include, but arenot limited to, elements of profile, account, transaction, or reportingdata characterizing corresponding ones of the customers of the financialinstitution, along with elements of delinquency data identifying andcharacterizing prior occurrences of default events associated with, orinvolving, the corresponding customers (e.g., that collectiveestablishes elements of “interaction data”). Further, the trainedmachine-learning or artificial-intelligence process (e.g., the trainedgradient-boosted, decision-tree process described herein) may furtheringest input datasets associated with one or more customers of thefinancial institution, and based on an application of the trainedgradient-boosted, decision-tree process to the input datasets, the oneor more FI computing systems may generate elements of output dataindicative of a likelihood of an occurrence of a default event involvingcorresponding ones of the customers during a future temporal interval,such as, but not limited to, a twelve-month period interval disposedsubsequent to a prediction date.

As described herein, and for the customer holding the credit cardaccount, a default event may occur when the credit card account isassociated with a past due balance (e.g., that accrues due to scheduledpayments missed, or delayed, by the customer), and when the a past-duebalance is associated with a corresponding past-due interval (e.g., asdefined by the number of scheduled payments missed, or delayed, by thecustomer) that exceeds a predetermined threshold time period (e.g.,ninety days, etc.). An occurrence of a default event may also beassociated with an inability of the financial institution to recoverall, or at least a portion of, an outstanding balance associated withthe credit-card account (e.g., based on a determination of the financialinstitution to “charge off” or write down the past-due balance on thecredit-card account, and to cease collection efforts involving the pastdue balance). For instance, the decision by the financial institution to“charge off” or write down the past-due balance on the credit-cardaccount may be triggered by the customer's declaration of, orassociation with, a personal or business bankruptcy.

Certain of these exemplary processes, which train and validate agradient-boosted, decision-tree process using customer-specific trainingand validation datasets associated with respective training andvalidation periods, and which apply the trained and validatedgradient-boosted, decision-tree process to additional customer-specificinput datasets, may enable the one or more of the FI computing systemsto predict, in real-time, a likelihood of an occurrence of an eventinvolving one or more customer of the financial institution, such as,but not limited to, the exemplary default event described herein, duringa predetermined, future temporal interval (e.g., via an implementationof one or more parallelized, fault-tolerant distributed computing andanalytical protocols across clusters of graphical processing units(GPUs) and/or tensor processing units (TPUs)). These exemplary processesmay, for example, be implemented in addition to, or as alternative to,processes through which the one or more FI computing systems computecustomer-specific scores indicative of a potential misuse of financial,products or services during a current temporal interval or thatcharacterize a relationship between the financial institution and acorresponding customer during the current temporal interval.

Further, and as described herein, certain of the exemplary processesdescribed herein provide, to the financial institution, a real-timeindication of the likelihood of a future default event (e.g., during thefuture temporal interval) involving one or more customers, which mayinform a determination of not only an initial set of terms andconditions associated with a newly issued credit product, but may alsoinform decisions, by the financial institution, to approve or declinerequests for modifications to an initial set of terms and conditions, orto authorize a transaction involving the issued credit product, as wellas decisions, by the financial institution, to suspend, close, orsubsequently reissue the credit product, and decisions to implement oneor more collection processes or strategies involving the credit product.By way of example, a customer may request, via a digital channel (e.g.,through a mobile application executed at customer device, etc.) or anin-person branch appointment, that the financial institution increase aninitial credit limit established for a credit-card account. Based on animplementation of any of the exemplary processes described herein, theone or more FI computing systems may generate, in real-time andcontemporaneously with the requested credit-limit increase, output dataindicative of an indication of the likelihood of a future default eventinvolving the customer and the credit-card account, and the financialinstitution may elect to approve the requested credit-limit increase(e.g., to issue a “positive” decision) or alternatively, to decline therequested credit-limit increase (e.g., to issue an “adverse” decision).

Further, and in addition to an adverse decision that declines thecredit-limit increase requested by the customer (e.g., based on thegenerated output data characterizing the likelihood of the futuredefault event), the financial institution may also provision, to thecustomer, information that explains the adverse decision and identifiesone or more of the factors that resulted in the decision of thefinancial institution to decline the requested credit-limit increase. Insome instances, however, the one or more factors identified within theprovisioned information may include data characterizing one or morecoarse metrics of the customer's use or misuse of the credit-cardaccount, and additionally, or alternatively, the customer's interactionof the financial institution, and may not reflect an impact of each, ora selected subset, of the feature values of a corresponding,customer-specific input dataset on the output data derived from anapplication of the trained gradient-boosted, decision-tree processes tothe customer-specific input dataset. By way of example, the provisionedinformation may include one or more reasons for the adverse decision,which may be generated manually by a representative of the financialinstitution, or programmatically by the one or more FI computingsystems, based on one or more product- or customer-specific rules orreasons, or which may be generated by representatives of the financialinstitution based on, among other thing, an experience or intuition ofthe representative.

In some instances, described herein, the one or more FI computingsystems may perform operations that apply one, or more, explainabilityprocesses to the customer-specific input dataset, and based on theapplication of the one or more explainability processes to thecustomer-specific input dataset, the one or more FI computing systemsmay generate elements of natural language that characterize a causalrelationship between the corresponding feature values of thecustomer-specific input dataset and the predicted output data generatedthrough an application of the trained gradient-boosted, decision-treeprocess to the customer-specific input dataset. By way of example, theone or more FI computing systems may train an explainability process(e.g., a Shapley-splitter process, as described herein) against elementsof one or more validation datasets associated with the trainedgradient-boosted, decision-tree process to generate, for each, or aselected subset, of the feature values of customer-specific inputdataset, corresponding elements of natural language that characterize acausal relationship between the corresponding feature value and thepredicted output data. The one or more FI computing systems may applythe trained explainability process to the elements (e.g., the featurevalues) of the customer-specific input data set concurrently with theapplication of the trained, gradient-boosted decision-tree process tothat customer-specific input data (e.g., concurrently with, orsubsequent to, inferencing), and as described herein, the elements ofnatural language may characterize an impact of the at least one featurevalue on an adverse decision associated with the output data (e.g., theadverse decision that declines the requested credit-limit increase, asdescribed herein), in a manner readily apparent to, and appreciable by,both representatives and customers of the financial institution.

Certain of the exemplary processes described herein, which adaptivelyand dynamically associate one or more feature values of acustomer-specific input dataset with a corresponding impact of the oneor more feature values of a predicted output of a trained, artificialintelligence or machine learning process, and that generate elements ofnatural characterizing an adverse decision associated with predictedoutput based on the corresponding impact, may be implemented by the oneor more FI computing systems in addition to, or as an alternate to,conventional mechanisms for developing rationales for adverse decisionsbased on inflexible, fixed rules or based on an intuition or anexperience of a representative of the financial institution. Further,certain of these exemplary reason generation processes described herein,which link the dynamic and programmatic generation of the elements ofnatural language, e.g., the “adverse reasons,” with the trained,artificial intelligence or machine learning process, may enhance anexplainability of the trained, artificial intelligence or machinelearning process and its role in the decision-making processes of thefinancial institution.

Further, although the exemplary reason generation processes aredescribed with respect to a trained, artificial intelligence or machinelearning process that predict a likelihood of a future default eventinvolving one or more customers, the disclosed embodiments are notlimited to this exemplary trained, gradient-boosted, decision-treeprocess, and in other examples, one or more of the exemplaryexplainability process described herein, such as, but not limited to,the trained Shapley-splitter process or explainability processesassociated with local partial dependency plots, may be applied to thevalidation data sets or predicted output data associated with anyadditional, or alternate, trained, gradient-boosted decision treeprocesses (or other trained, artificial intelligence or machine learningprocesses) and may generate elements of natural language thatcharacterize causal relationship between the corresponding featurevalues of a customer-specific input dataset and the predicted outputdata concurrently with, or subsequent to, inferencing.

A. Exemplary Techniques for Training Gradient-Boosted, Decision TreeProcesses in a Distributed Computing Environment

FIGS. 1A and 1B illustrate components of an exemplary computingenvironment 100, in accordance with some exemplary embodiments. Forexample, as illustrated in FIG. 1A, environment 100 may include one ormore source systems 102, such as, but not limited to, source system 102Aand source system 102B, and one or more computing systems associatedwith, or operated by, a financial institution, such as a transactionsystem 110 and a financial institution (FI) computing system 130. Insome instances, each of source systems 102 (including source system 102Aand source system 102B), transaction system 110, and FI computing system130, may be interconnected through one or more communications networks,such as communications network 120. Examples of communications network120 include, but are not limited to, a wireless local area network(LAN), e.g., a “Wi-Fi” network, a network utilizing radio-frequency (RF)communication protocols, a Near Field Communication (NFC) network, awireless Metropolitan Area Network (MAN) connecting multiple wirelessLANs, and a wide area network (WAN), e.g., the Internet.

In some examples, each of source systems 102 (including source system102A and source system 102B), transaction system 110, and FI computingsystem 130 may represent a computing system that includes one or moreservers and tangible, non-transitory memories storing executable codeand application modules. Further, the one or more servers may eachinclude one or more processors, which may be configured to executeportions of the stored code or application modules to perform operationsconsistent with the disclosed embodiments. For example, the one or moreprocessors may include a central processing unit (CPU) capable ofprocessing a single operation (e.g., a scalar operations) in a singleclock cycle. Further, each of source systems 102 (including sourcesystem 102A and source system 102B), transaction system 110, and FIcomputing system 130 may also include a communications interface, suchas one or more wireless transceivers, coupled to the one or moreprocessors for accommodating wired or wireless internet communicationwith other computing systems and devices operating within environment100.

Further, in some instances, source systems 102 (including source system102A and source system 102B), transaction system 110, and FI computingsystem 130 may each be incorporated into a respective, discretecomputing system. In additional, or alternate, instances, one or more ofsource systems 102 (including source system 102A and source system102B), transaction system 110, and FI computing system 130 maycorrespond to a distributed computing system having a plurality ofinterconnected, computing components distributed across an appropriatecomputing network, such as communications network 120 of FIG. 1A. Forexample, FI computing system 130 may correspond to a distributed orcloud-based computing cluster associated with and maintained by thefinancial institution, although in other examples, FI computing system130 or transaction system 110 may correspond to a publicly accessible,distributed or cloud-based computing cluster, such as a computingcluster maintained by Microsoft Azure™, Amazon Web Services™, GoogleCloud™, or another third-party provider.

In some instances, FI computing system 130 may include a plurality ofinterconnected, distributed computing components, such as thosedescribed herein (not illustrated in FIG. 1A), which may be configuredto implement one or more parallelized, fault-tolerant distributedcomputing and analytical processes (e.g., an Apache Spark™ distributed,cluster-computing framework, a Databricks™ analytical platform, etc.).Further, and in addition to the CPUs described herein, the distributedcomputing components of FI computing system 130 may also include one ormore graphics processing units (GPUs) capable of processing thousands ofoperations (e.g., vector operations) in a single clock cycle, andadditionally, or alternatively, one or more tensor processing units(TPUs) capable of processing hundreds of thousands of operations (e.g.,matrix operations) in a single clock cycle. Through an implementation ofthe parallelized, fault-tolerant distributed computing and analyticalprotocols described herein, the distributed computing components of FIcomputing system 130 may perform any of the exemplary processesdescribed herein to ingest elements of data associated with thecustomers of the financial institution, including elements oftransaction data characterizing purchase transaction involving thesecustomers, to preprocess the ingested data element and characterize, inreal-time, trends or patterns in the customers' purchase transactions,and to store the preprocessed data elements within an accessible datarepository (e.g., within a portion of a distributed file system, such asa Hadoop distributed file system (HDFS)).

Further, and through an implementation of the parallelized,fault-tolerant distributed computing and analytical protocols describedherein, the distributed components of FI computing system 130 mayperform operations in parallel that not only train adaptively a machinelearning or artificial intelligence process (e.g., the gradient-boosted,decision-tree process described herein) using corresponding training andvalidation datasets extracted from temporally distinct subsets of thepreprocessed data elements, but also apply the trained machine learningor artificial intelligence process to customer-specific input datasetsand generate, in real time, elements of output data indicative of alikelihood of an occurrence of a default event involving correspondingones of the customers during a future temporal interval, such atwelve-month interval subsequent to a prediction date. Theimplementation of the parallelized, fault-tolerant distributed computingand analytical protocols described herein across the one or more GPUs orTPUs included within the distributed components of FI computing system130 may, in some instances, accelerate the training, and thepost-training deployment, of the machine-learning andartificial-intelligence process when compared to a training anddeployment of the machine-learning and artificial-intelligence processacross comparable clusters of CPUs capable of processing a singleoperation per clock cycle.

Referring back to FIG. 1A, each of source systems 102 may maintain,within corresponding tangible, non-transitory memories, a datarepository that includes confidential data associated with the customersof the financial institution. For example, source system 102A may beassociated with, or operated by, the financial institution, and maymaintain, within one or more tangible, non-transitory memories, a sourcedata repository 103 that includes elements of source data 104identifying or characterizing customers of the financial institution andinteractions between these customers and the financial institution, suchas, but are not limited to, customer profile data 104A, account data104B, and delinquency data 104C. In some instances, customer profiledata 104A may include a plurality of data records associated with, andcharacterizing, corresponding ones of the customers of the financialinstitution. By way of example, and for a particular customer of thefinancial institution, the data records of customer profile data 104Amay include, but are not limited to, one or more unique customeridentifiers (e.g., an alphanumeric character string, such as a logincredential, a customer name, etc.), residence data (e.g., a streetaddress, etc.), other elements of contact data (e.g., a mobile number,an email address, etc.), values of demographic parameters thatcharacterize the particular customer (e.g., ages, occupations, maritalstatus, etc.), and other data characterizing the relationship betweenthe particular customer and the financial institution.

Account data 104B may also include a plurality of data records thatidentify and characterize one or more financial products or financialinstruments issued by the financial institution to corresponding ones ofthe customers. For example, the data records of account data 104B mayinclude, for each of the financial products issued to corresponding onesof the customers, one or more identifiers of the financial product orinstrument (e.g., an account number, expiration data,card-security-code, etc.), one or more unique customer identifiers(e.g., an alphanumeric character string, such as a login credential, acustomer name, etc.), and additional information characterizing abalance or current status of the financial product or instrument (e.g.,payment due dates or amounts, delinquent accounts statuses, etc.).Examples of these financial products or financial instruments mayinclude, but are not limited to, one or more deposit accounts issued tocorresponding ones of the customers (e.g., a savings account, a checkingaccount, etc.), one or more secured or unsecured credit products (e.g.,a secured or unsecured a credit-card account, etc.), one or morebrokerage or retirements accounts issued to corresponding ones of thecustomers by the financial institutions, and one or more secured creditproducts issued to corresponding ones of the customers by the financialinstitution (e.g., a home mortgage, a home-equity line-of-credit(HELOC), an auto loan, etc.).

Further, delinquency data 104C may include data records that identifyand characterize occurrences of default events involving customers ofthe financial institution and corresponding financial products orfinancial instruments issued by the financial institution, such as thedefault events associated with the credit-card accounts describedherein. In some instances, each of the data records of delinquency data104C may associate with a corresponding occurrence of a default event,and may include, for the corresponding occurrence of the default event,a unique identifier of a corresponding customer (e.g., an alphanumericidentifier or login credential, a customer name, etc.), temporal datacharacterizing of the corresponding occurrence of the default event(e.g., a time or date, etc.), information identifying one or morefinancial products or financial instruments associated with thecorresponding occurrence of the default event (e.g., a portion of atokenized account number for a credit-card account, etc.), andadditionally, or alternatively, information characterizing thecorresponding occurrence of the default event (e.g., an event type, suchas the past-due balance on the credit-card account, the bankruptcy, orthe write-down described herein, etc.).

The disclosed embodiments are, however, not limited to these exemplaryelements of customer profile data 104A, account data 104B, ordelinquency data 104C. In other instances, the data records of sourcedata 104 may include any additional or alternate elements of data thatidentify and characterize the customers of the financial institution andtheir relationships or interactions with the financial institution,financial products issued to these customers by the financialinstitution, and any additional, or alternate, informationcharacterizing prior occurrences of default events involving customer ofthe financial institution. Further, although stored in FIG. 1A withindata repositories maintained by source system 102A, the exemplaryelements of customer profile data 104A, account data 104B, anddelinquency data 104C may be maintained by any additional or alternatecomputing system associated with the financial institution, including,but not limited to, within one or more tangible, non-transitory memoriesof FI computing system 130.

Source system 102B may be associated with, or operated by, one or morejudicial, regulatory, governmental, or reporting entities external to,and unrelated to, the financial institution, and source system 102B maymaintain, within the corresponding one or more tangible, non-transitorymemories, a source data repository 106 that includes one or moreelements of source data 108 generated by the judicial, regulatory,governmental, or regulatory entities described herein, such asadditional, or alternate, elements of credit-bureau data. In someinstances, source system 102B may be associated with, or operated by, areporting entity, such as a credit bureau, and source data 108 mayinclude data records that specify elements of credit-bureau data 108Aassociated with one or more customers of the financial institution. Insome instances, the elements of credit-bureau data 108A for a particularone of the customers of the financial institution may include, but arenot limited to, a unique identifier of the particular customer (e.g., analphanumeric identifier or login credential, a customer name, etc.),information identifying one or more financial products currently orpreviously held by the particular customer (e.g., one or more of thefinancial products or payment instruments described herein, financialproducts issued by other financial institutions, etc.), and informationidentifying one or more of a history of payments associated with thesefinancial products, negative events associated with the particularcustomer (e.g., missed payments, collections, repossessions, etc.), orcredit inquiries involving the particular customer (e.g., inquiries bythe financial institution, other financial institutions or businessentities, etc.).

Further, and as illustrated in FIG. 1A, transaction system 110 may alsobe associated with, or operated by, the financial institution, and maymaintain, within the corresponding one or more tangible, non-transitorymemories, a transaction data store 112 having one or more transactiondata records that maintain elements of transaction data 114 identifying,and characterizing, transactions initiated by, and involving, customersof the financial institution. Each of the transactions may, for example,be initiated by a customer of the financial institution and involve acorresponding counterparty (e.g., a merchant, retailer, or otherbusiness that offers products or services for sale), and may be fundedby a corresponding one of the financial products or instruments held bythat customer, such as, but not limited to, the credit productsdescribed herein.

In some instances, not illustrated in FIG. 1A, transaction system 110may, via a secure programmatic channel of communications, portions oftransaction data 114 from the one or more additional computing systemsoperating within environment 100 in real-time data and on a continuousstreaming basis, on in batch form in accordance with a predeterminedtemporal schedule (e.g., on a daily basis, a monthly basis, etc.). Theone or more additional computing systems may, for example, be associatedwith a transaction processing network, such as, but not limited to, apayment rail that clears and settles purchase transactions funded viacorresponding credit-card accounts. Further, in some instances, the oneor more of the additional computing systems may be associated withreal-time payment rail that processes and facilitates real-time payment(RTP) transactions between counterparties (e.g., via payment messagesstructured in accordance with an ISO-20022 messaging standard).Additionally, or alternatively, one or more of the additional computingsystems may be associated with a mobile payment rail that processescertain peer-to-peer (P2P) transactions between the customers of thefinancial institutions and corresponding counterparties, or an automatedclearing house (ACH) that also process and facilitate certain of the P2Ptransactions described herein, along with electronic funds transfer(EFT) transactions between the customers of the financial institutionsand the corresponding counterparties.

Referring back to FIG. 1A, FI computing system 130 may performoperations that establish and maintain one or more centralized datarepositories within corresponding ones of the tangible, non-transitorymemories. For example, as illustrated in FIG. 1A, FI computing system130 may establish an aggregated data store 132, which maintains, amongother things, elements of the customer profile, account, transaction,delinquency, and credit-bureau data associated with one or more of thecustomers of the financial institution, which may be ingested by FIcomputing system 130 (e.g., from one or more of source systems 102and/or from transaction system 110) using any of the exemplary processesdescribed herein. Aggregated data store 132 may, for instance,correspond to a data lake, a data warehouse, or another centralizedrepository established and maintained, respectively, by the distributedcomponents of FI computing system 130, e.g., through a Hadoop™distributed file system (HDFS).

For example, FI computing system 130 may execute one or more applicationprograms, elements of code, or code modules that, in conjunction withthe corresponding communications interface, establish a secure,programmatic channel of communication with each of source systems 102,including source system 102A and source system 102B, across network 120,and may perform operations that access and obtain all, or a selectedportion, of the elements of customer profile, account, delinquency,and/or reporting data maintained by corresponding ones of source systems102. As illustrated in FIG. 1A, source system 102A may performoperations that obtain all, or a selected portion, of source data 104,including the data records of customer profile data 104A, account data104B, and delinquency data 104C, from source data repository 103, andtransmit the obtained portions of source data 104 across network 120 toFI computing system 130. Further, source system 102B may also performoperations that obtain all, or a selected portion, of source data 108,including the data records of credit-bureau data 108A, from externaldata repository 106, and transmit the obtained portions of source data108 across network 120 to FI computing system 130. In some instances,each of source systems 102, including source system 102A and sourcesystem 102B, may perform operations that transmit respective portions ofsource data 104 and source data 108 across network 120 to FI computingsystem 130 in batch form and in accordance with a predetermined temporalschedule (e.g., on a daily basis, on a monthly basis, etc.), or inreal-time on a continuous, streaming basis.

Further, the one or more executed application programs, elements ofcode, or code modules may also cause FI computing system 130 to performoperations that, in conjunction with the corresponding communicationsinterface, establish a secure, programmatic channel of communicationwith transaction system 110 across network 120, and may performoperations that access and obtain all, or a selected portion, of thetransaction data 114 maintained within transaction data store 112. Forexample, transaction system 110 may access transaction data store 112,and perform operations that transmit all, or a selected portion, oftransaction data 114 across network 120 to FI computing system 130. Asdescribed herein, transaction system 110 may perform operations thattransmit portions of transaction data 114 across network 120 to FIcomputing system 130 in real-time on a continuous streaming basis (e.g.,upon receipt of transaction data 114 at transaction system 110) or inaccordance with a predetermined temporal schedule (e.g., on an hourlybasis, on a daily basis, on a monthly basis, etc.).

A programmatic interface established and maintained by FI computingsystem 130, such as application programming interface (API) 134, mayreceive the portions of source data 104 (including the data records ofcustomer profile data 104A, account data 104B, and delinquency data104C) from source system 102A, the portions of source data 108(including the data records of credit-bureau data 108A) from sourcesystem 102B, and portions of transaction data 114 from transactionsystem 110. The received portions of source data 104, source data 108,and transaction data 114 may collectively represent element ofinteraction data (e.g., interaction data 135 of FIG. 1A, and API 134 mayroute the portions of source data 104, source data 108, and transactiondata 114 to a data ingestion engine 136 executed by the one or moreprocessors of FI computing system 130. In some instances, the portionsof source data 104 and source data 108 (and the additional, oralternate, portions of the customer profile, account, delinquency, orreporting data), and/or transaction data 114, may be encrypted via acorresponding encryption key (e.g., a public cryptographic key of FIcomputing system 130), and executed data ingestion engine 136 mayperform operations that decrypt each of the encrypted portions of sourcedata 104 and source data 108 (and the additional, or alternate, portionsof the customer profile, account, delinquency, or reporting data),and/or the portions of transaction data 114, using a correspondingdecryption key, e.g., a private cryptographic key associated with FIcomputing system 130. Executed data ingestion engine 136 may alsoperform operations that store the portions of source data 104 (includingthe data records of customer profile data 104A, account data 104B, anddelinquency data 104C), source data 108 (including the data records ofcredit-bureau data 108A), and transaction data 114 within aggregateddata store 132, e.g., as ingested customer data 138.

In some instances, a pre-processing engine 140 executed by the one ormore processors of FI computing system 130 may access ingested customerdata 138, and perform any of the exemplary data pre-processingoperations described herein to selectively aggregate, filter, andprocess portions of the elements of ingested customer data 138, and togenerate consolidated data records 142 that characterize correspondingones of the customers, their interactions with the financial institutionand with other financial institutions, and any associated default eventsduring a corresponding temporal interval associated with the ingestionof source data 104, source data 108, and transaction data 114 byexecuted data ingestion engine 136. By way of example, executedpre-processing engine 140 may access the data records of profile data104A, account data 104B, delinquency data 104C, credit-bureau data 108A,and in some instances, transaction data 114 (e.g., as maintained withiningested customer data 138). As described herein, each of the accesseddata records may include an identifier of corresponding customer of thefinancial institution, such as a customer name or an alphanumericcharacter string, and executed pre-processing engine 140 may performoperations that map each of the accessed data records to a customeridentifier assigned to the corresponding customer by FI computing system130. For instance, FI computing system 130 may assign a unique,alphanumeric customer identifier to each customer, and executedpre-processing engine 140 may perform operations that parse the accesseddata records, obtain each of the parsed data records that identifies thecorresponding customer using a customer name, and replace that customername with the corresponding alphanumeric customer identifier.

Executed pre-processing engine 140 may also perform operations thatassign a temporal identifier to each of the accessed data records, andthat augment each of the accessed data records to include the newlyassigned temporal identifier. In some instances, the temporal identifiermay associate each of the accessed data records with a correspondingtemporal interval, which may be indicative of or reflect a regularity ora frequency at which FI computing system 130 ingests the elements ofsource data 104 and source data 108 from corresponding ones of sourcesystems 102 and/or transaction data 114 from transaction system 110. Forexample, executed data ingestion engine 136 may receive elements ofconfidential customer data from corresponding ones of source systems 102on a monthly basis (e.g., on the final day of the month), and inparticular, may receive and store the elements of source data 104 andsource data 108 from corresponding ones of source systems 102 on, forexample, Nov. 30, 2021. In some instances, executed pre-processingengine 140 may generate a temporal identifier associated with theregular, monthly ingestion of source data 104 and source data 108 onNov. 30, 2021 (e.g., “2021-11-30”), and may augment the accessed datarecords of profile data 104A, account data 104B, delinquency data 104C,credit-bureau data 108A, and/or transaction data 114 to include thegenerated temporal identifier. The disclosed embodiments are, however,not limited to temporal identifiers reflective of a regular, monthlyingestion of source data 104 and source data 108 by FI computing system130, and in other instances, executed pre-processing engine 140 mayaugment the accessed data records to include temporal identifiersreflective of any additional, or alternative, temporal interval duringwhich FI computing system 130 ingests the elements of source data 104,source data 108, and transaction data 114.

In some instances, executed pre-processing engine 140 may performfurther operations that, for a particular customer of the financialinstitution during the temporal interval (e.g., represented by a pair ofthe customer and temporal identifiers described herein), obtain one ormore data records of profile data 104A, account data 104B, delinquencydata 104C, credit-bureau data 108A, and/or transaction data 114 thatinclude the pair of customer and temporal identifiers. Executedpre-processing engine 140 may perform operations that consolidate theone or more obtained data records and generate a corresponding one ofconsolidated data records 142 that includes the customer identifier andtemporal identifier, and that is associated with, and characterizes, theparticular customer of the financial institution across the temporalintervals. By way of example, executed pre-processing engine 140 mayconsolidate the obtained data records, which include the pair ofcustomer and temporal identifiers, through an invocation of anappropriate Java-based SQL “join” command (e.g., an appropriate “inner”or “outer” join command, etc.). Further, executed pre-processing engine140 may perform any of the exemplary processes described herein togenerate another one of consolidated data records 142 for eachadditional, or alternate, customer of the financial institution duringthe temporal interval (e.g., as represented by a corresponding customeridentifier and the temporal interval).

Executed pre-processing engine 140 may perform operations that storeeach of consolidated data records 142 within the one or more tangible,non-transitory memories of FI computing system 130, such as withinconsolidated data store 144. Consolidated data store 144 may, forinstance, correspond to a data lake, a data warehouse, or anothercentralized repository established and maintained, respectively, by thedistributed components of FI computing system 130, e.g., through aHadoop™ distributed file system (HDFS). In some instances, and asdescribed herein, consolidated data records 142 may include a pluralityof discrete data records, and each of these discrete data records may beassociated with, and may maintain data characterizing, a correspondingone of the customers of the financial institution during thecorresponding temporal interval (e.g., a month-long interval extendingfrom Nov. 1, 2021, to Nov. 30, 2021). For example, and for a particularcustomer of the financial institution, discrete data record 142A ofconsolidated data records 142 may include a customer identifier 146 ofthe particular customer (e.g., an alphanumeric character string“CUSTID”), a temporal identifier 148 of the corresponding temporalinterval (e.g., a numerical string “2021-11-30”), and consolidated dataelements 150 of customer profile, account, delinquency, credit-bureau,and/or transaction data that characterize the particular customer duringthe corresponding temporal interval (e.g., as consolidated from the datarecords of profile data 104A, account data 104B, delinquency data 104C,credit-bureau data 108A, and/or transaction data 114 ingested by FIcomputing system 130 on Nov. 30, 2021).

In some instances, consolidated data elements 150 include alsoaggregated values of customer profile, account, delinquency,credit-bureau, and/or transaction parameters that characterize abehavior of the particular customer during the temporal intervalextending from Nov. 1, 2021, to Nov. 30, 2021. For example, executedpre-processing engine 140 may process the data records of account data104B (e.g., as maintained within ingested customer data 138) to computeaggregate values of account parameters that include, but are not limitedto, an average balance of one or more accounts held by the particularcustomer, a total number of withdrawals of funds from, or deposits offunds into, one or more of the accounts held the particular customer, ora total value of the funds withdrawn from, or deposited into, the one ormore of the accounts during the month-long interval. Additionally, insome examples, executed pre-processing engine 140 may process the datarecords of transaction data 114 (e.g., as maintained within ingestedcustomer data 138) to compute aggregate values of transaction parametersthat include, but are not limited to, an aggregate value of transactionsinitiated, cleared and settled during month-long interval, an averagedaily value of the initiated, cleared and settled transactions, or anaggregate or average daily value of those initiated, cleared, andsettled transactions that involve a particular payment instrument, or aparticular counterparty. The disclosed embodiments are, however, notlimited to these exemplary aggregate values of account or transactionparameters, and in other examples, executed pre-processing engine 140may compute any additional or alternate aggregated values of account ortransaction parameters the characterize the behavior of particularcustomer.

Further, in some instances, consolidated data store 144 may maintaineach of consolidated data records 142, which characterize correspondingones of the customers, their interactions with the financial institutionand with other financial institutions, and any associated default eventsduring the temporal interval, in conjunction with additionalconsolidated data records 152. Executed pre-processing engine 140 mayperform any of the exemplary processes described herein to generate eachof the additional consolidated data records 152, including based onelements of profile, account, delinquency, credit-bureau, and/ortransaction data ingested from source systems 102 or transaction system110 during the corresponding prior temporal intervals.

Each of additional consolidated data records 152 may also include aplurality of discrete data records that are associated with andcharacterize a particular one of the customers of the financialinstitution during a corresponding one of the prior temporal intervals.For example, as illustrated in FIG. 1A, additional consolidated datarecords 152 may include one or more discrete data records, such asdiscrete data record 154, associated with a prior temporal intervalextending from Sep. 1, 2021, to Sep. 30, 2021. For the particularcustomer, discrete data record 154 may include a customer identifier 156of the particular customer (e.g., an alphanumeric character string“CUSTID”), a temporal identifier 158 of the prior temporal interval(e.g., a numerical string “2021-10-31”), consolidated elements 160 ofcustomer profile, account, delinquency, credit-bureau, and/ortransaction data that characterize the particular customer during theprior temporal interval extending from Oct. 1, 2021, to Oct. 31, 2021(e.g., as consolidated from the data records ingested by FI computingsystem 130 on Oct. 31, 2021). In some instances, consolidated dataelements 160 may also elements of aggregated values of customer profile,account, delinquency, credit-bureau, and/or transaction parameters thatcharacterize a behavior of the particular customer during the priortemporal interval, such as, but not limited to, the exemplary aggregatedvalues described herein.

The disclosed embodiments are, however, not limited to the exemplaryconsolidated data records described herein, or to the exemplary temporalintervals described herein. In other examples, FI computing system 130may generate, and the consolidated data store 144 may maintain anyadditional or alternate number of discrete sets of consolidated datarecords, having any additional or alternate composition, that would beappropriate to the elements of customer profile, account, delinquency,credit-bureau, and/or transaction data ingested by FI computing system130 at the predetermined intervals described herein. Further, in someexamples, FI computing system 130 may ingest elements of customerprofile, account, delinquency, credit-bureau, and/or transaction datafrom source systems 102 or transaction system 110 at any additional, oralternate, fixed or variable temporal interval that would be appropriateto the ingested data or to the training of the machine learning orartificial intelligence processes described herein, including acontinuous, real-time ingestion of the elements of customer profile,account, delinquency, or credit-bureau data.

In some instances, FI computing system 130 may perform operations thattrain adaptively a machine-learning or artificial-intelligence processto predict a likelihood of an occurrence of a default event involvingone or more customers of the financial institution during a futuretemporal interval using training datasets associated with a first priortemporal interval (e.g., a “training” interval), and using validationdatasets associated with a second, and distinct, prior temporal interval(e.g., an out-of-time “validation” interval). As described herein, andfor a particular customer the financial institution that holds a creditcard account, a default event may occur when the credit card account isassociated with a past due balance (e.g., that accrues due to scheduledpayments missed, or delayed, by the particular customer), and when thepast-due balance is associated with a corresponding past-due interval(e.g., as defined by the number of scheduled payments missed, ordelayed, by the customer) that exceeds a predetermined threshold timeperiod (e.g., ninety days, etc.). An occurrence of a default event mayalso be associated with an inability of the financial institution torecover all, or at least a portion of, an outstanding balance associatedwith the credit-card account (e.g., based on a determination of thefinancial institution to “charge off” or write down the past-due balanceon the credit-card account, and to cease collection efforts involvingthe past due balance). For instance, the decision by the financialinstitution to “charge off” or write down the past-due balance on thecredit-card account may be triggered by the particular customer'sdeclaration of, or association with, a personal or business bankruptcy.

Further, and as described herein, the machine-learning orartificial-intelligence process may include an ensemble or decision-treeprocess, such as a gradient-boosted decision-tree process (e.g., theXGBoost process), and the training and validation datasets may include,but are not limited to, values of adaptively selected features obtained,extracted, or derived from the consolidated data records maintainedwithin consolidated data store 144, e.g., from data elements maintainedwithin the discrete data records of consolidated data records 142 or theadditional consolidated data records 152. By way of example, the valuesof adaptively selected features of the training and validation datasetsmay be obtained, extracted, or derived from the consolidated elements ofcustomer profile data, account data, delinquency data, credit-bureaudata, and in some instances, transaction maintained within theconsolidated data records of consolidated data store 144. The adaptiveselected feature values may also include one, or more, of the elementsof aggregated customer profile, account, delinquency, credit-bureau, ortransaction data that characterize the customers of the financialinstitution during respective ones of the training and validationintervals.

For example, the distributed computing components of FI computing system130 (e.g., that include one or more GPUs or TPUs configured to operateas a discrete computing cluster) may perform any of the exemplaryprocesses described herein to train the machine learning or artificialintelligence process (e.g., the gradient-boosted, decision-tree process)in parallel through an implementation of one or more parallelized,fault-tolerant distributed computing and analytical processes. Based onan outcome of these training processes, FI computing system 130 maygenerate process coefficients, parameters, thresholds, and other datathat collectively specify the trained machine learning or artificialintelligence process, and may store the generated process coefficients,parameters, thresholds, and other data within a portion of the one ormore tangible, non-transitory memories, e.g., within consolidated datastore 144.

For example, and with reference to FIG. 1B, a training engine 162executed by the one or more processors of FI computing system 130 mayaccess the consolidated data records maintained within consolidated datastore 144, such as, but not limited to, the discrete data records ofconsolidated data records 142 or additional consolidated data records152. As described herein, each of the consolidated data records, such asdiscrete data record 142A of consolidated data records 142 or discretedata record 154 of additional consolidated data records 152, may includea customer identifier of a corresponding one of the customers of thefinancial institution (e.g., customer identifiers 146 and 156 of FIG.1A) and a temporal identifier that associates the consolidated datarecord with a corresponding temporal interval (e.g., temporalidentifiers 148 and 158 of FIG. 1A). Each of the accessed consolidateddata records may also include consolidated elements of customer profile,account, delinquency, credit-bureau, and/or transaction data thatcharacterize the corresponding one of the customers during thecorresponding temporal interval (e.g., consolidated data elements 150and 160 of FIG. 1A), and aggregated values of customer profile, account,delinquency, credit-bureau, and/or transaction parameters thatcharacterize the corresponding one of the customers during thecorresponding temporal interval (e.g., aggregated data elements 151 and161 of FIG. 1A).

In some instances, executed training engine 162 may parse the accessedconsolidated data records, and based on corresponding ones of thetemporal identifiers, determine that the consolidated elements ofcustomer profile, account, delinquency, credit-bureau data, and/ortransaction data characterize the corresponding customers across a rangeof prior temporal intervals. Further, executed training engine 162 mayalso perform operations that decompose the determined range of priortemporal intervals into a corresponding first subset of the priortemporal intervals (e.g., the “training” interval described herein) andinto a corresponding second, subsequent, and disjoint subset of theprior temporal intervals (e.g., the “validation” interval describedherein). For example, as illustrated in FIG. 1C, the range of priortemporal intervals (e.g., shown generally as Δt along timeline 163 ofFIG. 1C) may be bounded by, and established by, temporal boundariest_(i) and t_(f). Further, the decomposed first subset of the priortemporal intervals (e.g., shown generally as training intervalΔt_(training) along timeline 163 of FIG. 1C) may be bounded by temporalboundary t_(i) and a corresponding splitting point t_(split), and thedecomposed second subset of the prior temporal intervals (e.g., showngenerally as validation interval Δt_(validation) along timeline 163 ofFIG. 1C) may be bounded by splitting point t_(split) and temporalboundary t_(f).

Referring back to FIG. 1B, executed training engine 162 may generateelements of splitting data 164 that identify and characterize thedetermined temporal boundaries of the consolidated data recordsmaintained within consolidated data store 144 (e.g., temporal boundariest_(i) and t_(f)) and the range of prior temporal intervals establishedby the determined temporal boundaries Further, the elements of splittingdata 164 may also identify and characterize the splitting point (e.g.,the splitting point t_(split) described herein), the first subset of theprior temporal intervals (e.g., the training interval Δt_(training) andcorresponding boundaries described herein), and the second, andsubsequent subset of the prior temporal intervals (e.g., the validationinterval Δt_(validation) and corresponding boundaries described herein).As illustrated in FIG. 1B, executed training engine 162 may store theelements of splitting data 164 within the one or more tangible,non-transitory memories of FI computing system 130, e.g., withinconsolidated data store 144.

In some instances, each of the prior temporal intervals may correspondto a one-month interval, and executed training engine 162 may performoperations that establish adaptively the splitting point between thecorresponding temporal boundaries such that a predetermined firstpercentage of the consolidated data records are associated with temporalintervals (e.g., as specified by corresponding ones of the temporalidentifiers) disposed within the training interval, and such that apredetermined second percentage of the consolidated data records areassociated with temporal intervals (e.g., as specified by correspondingones of the temporal identifiers) disposed within the validationinterval. For example, the first predetermined percentage may correspondto seventy percent of the consolidated data records, and the secondpredetermined percentage may corresponding to thirty percent of theconsolidated data records, although in other examples, executed trainingengine 162 may compute one or both of the first and second predeterminedpercentages, and establish the decomposition point, based on the rangeof prior temporal intervals, a quantity or quality of the consolidateddata records maintained within consolidated data store 144, or amagnitude of the temporal intervals (e.g., one-month intervals, two-weekintervals, one-week intervals, one-day intervals, etc.).

In some examples, a training input module 166 of executed trainingengine 162 may perform operations that access the consolidated datarecords maintained within consolidated data store 144. Based on portionsof splitting data 164, executed training input module 166 may performoperations that parse the consolidated data records and determine: (i) afirst subset 168A of these consolidated data records that are associatedwith the training interval Δt_(training) and may be appropriate totraining adaptively the gradient-boosted, decision-tree process duringthe training interval; and a (ii) second subset 168B of theseconsolidated data records are associated with the validation intervalΔt_(validation) and may be appropriate to validating the trained,gradient-boosted, decision-tree process during the validation interval.

As described herein, FI computing system 130 may perform operations thatadaptively train a machine-learning or artificial-intelligence process(e.g., the gradient-boosted, decision-tree process described herein) topredict, during a current temporal interval, a likelihood of anoccurrence of a default event involving a customer during a futuretemporal interval using training datasets associated with the traininginterval, and using validation datasets associated with the validationinterval. For example, and as illustrated in FIG. 1D, the currenttemporal interval may be characterized by a temporal prediction pointt_(pred) along timeline 163, and the executed training engine 162 mayperform any of the exemplary processes described herein to trainmachine-learning or artificial-intelligence process (e.g., thegradient-boosted, decision-tree process described herein) to predict thelikelihood of occurrences of default events during a future, targettemporal interval Δt_(target) based on input datasets associated with acorresponding prior extraction interval Δt_(extract). The targettemporal interval Δt_(target) may be characterized by a predeterminedduration, such as, but not limited to, twelve months, and the priorextraction interval Δt_(extract) may be characterized by acorresponding, predetermined duration, such as, but not limited to, onemonth, three months, or six months.

Referring back to FIG. 1B, executed training input module 166 mayperform operations that access the consolidated data records maintainedwithin consolidated data store 144 (e.g., consolidated data records 142,152), and parse each of the consolidated data records to obtain acorresponding customer identifier (e.g., which associates with theconsolidated data record with a corresponding one of the customers ofthe financial institution) and a corresponding temporal identifier(e.g., which associated the consolidated data record with acorresponding temporal interval). For example, and based on the obtainedcustomer and temporal identifiers, executed training input module 166may generate sets of segmented data records associated withcorresponding ones of the customer identifiers (e.g., customer-specificsets of segmented data records), and within each set of segmented datarecords, executed training input module 166 may order the consolidateddata records sequentially in accordance with the obtained temporalinterval. Through these exemplary processes, executed training inputmodule 166 may generate sets of customer-specific, sequentially ordereddata records (e.g., data tables), which executed training input module166 may maintain locally within the consolidated data store 144 (notillustrated in FIG. 1B).

Executed training input module 166 may also perform operations thataugment the sequentially ordered data records within each of thecustomer-specific sets to include additional information characterizinga ground truth associated with the corresponding customer and temporalinterval (as established by the corresponding pair of customer andtemporal identifiers). For example, and for a particular one of thesequentially ordered data records, such as discrete data record 142A ofconsolidated data records 142, executed training input module 166 mayobtain customer identifier 146 (e.g., “CUSTID”), which identifies thecorresponding customer, and temporal identifier 148, which indicatesdata record 142A is associated with Nov. 30, 2021. Based on customeridentifier 146 and temporal identifier 148, executed training inputmodule 166 may access delinquency data 104C (e.g., as maintained withinaggregated data store 132 of FIG. 1A), and determine whether thecorresponding customer experienced a default event involving acorresponding credit-card account within the target intervalΔt_(target). For example, executed training input module 166 maydetermine that Nov. 31, 2021, as identified by the temporal identifier148, falls on or within the target interval Δt_(target), that thedelinquency data 140C identifies a default of the customer, and as suchthe corresponding customer experienced a default event within the targetinterval Δt_(target). Executed training input module 166 may performoperations that modify data record 142A by appending an element ofground-truth data indicative of the presence or absence of the defaultevent within the target interval Δt_(target) to consolidated dataelements 150. Executed training input module 166 may also perform any ofthe exemplary processes described herein to generate and append anappropriate element of ground-truth data to each additional, oralternate, one of the sequentially ordered data records within each ofthe customer-specific sets maintained within consolidated data store144.

Executed training input module 166 may also perform operations thatpartition the customer-specific sets of sequentially ordered datarecords into subsets suitable for training the gradient-boosted,decision-tree process (e.g., which may be maintained in first subset168A of consolidated data records within consolidated data store 144)and for validating the trained, gradient-boosted, decision-tree process(e.g., which may be maintained in second subset 168B of consolidateddata records within consolidated data store 144). By way of example,executed training input module 166 may access splitting data 164, andestablish the temporal boundaries for the training intervalΔt_(training) (e.g., temporal boundary t_(i) and splitting pointt_(split)) and the validation interval Δt_(training) (e.g., splittingpoint t_(split) and temporal boundary t_(f)). Further, executed traininginput module 166 may also parse each of the sequentially ordered datarecords of the customer-specific sets, access the corresponding temporalidentifier, and determine the temporal interval associated with the eachof sequentially ordered data records.

If, for example, executed training input module 166 were to determinethat the temporal interval associated with a corresponding one of thesequentially ordered data records is disposed within the temporalboundaries for the training interval Δt_(training), executed traininginput module 166 may determine that the corresponding data record may besuitable for training, and may perform operations that include thecorresponding data record within a portion of the first subset 168A(e.g., that store the corresponding data record within a portion ofconsolidated data store 144 associated with first subset 168A).Alternatively, if executed training input module 166 were to determinethat the temporal interval associated with a corresponding one of thesequentially ordered data records is disposed within the temporalboundaries for the validation interval Δt_(validation), executedtraining input module 166 may determine that the corresponding datarecord may be suitable for validation, and may perform operations thatinclude the corresponding data record within a portion of the secondsubset 168B (e.g., that store the corresponding data record within aportion of consolidated data store 144 associated with second subset168B). Executed training input module 166 may perform any of theexemplary processes described herein to determine the suitability ofeach additional, or alternate, one of the sequentially ordered datarecords of the customer-specific sets for training, or alternatively,validation, of the gradient-boosted, decision-tree process.

In some instances, executed training input module 166 may also performoperations that filter the consolidated data records of first subset168A and second subset 168B in accordance with one or more filtrationcriteria. By way of example, the one or more filtration criteria maycause executed training input module 166 to perform operations thatexclude, from first subset 168A and second subset 168B, a consolidateddata record of any customer associated with an occurrence of a defaultevent involving a credit-card account during, or prior to, the temporalinterval associated with the corresponding temporal identifier (e.g., acredit card account associated with a past-due balance having acorresponding past-due interval that exceeds a predetermined thresholdtime period, ninety days, or a past-due balance charged-off or writtendown by the financial institution). The one or more filtration criteriamay also cause executed training input module 166 to perform operationsthat exclude, from first subset 168A and second subset 168B, aconsolidated data record of any customer holding a credit-card accountissued by the financial institution within a predetermined priortemporal interval (e.g., three months, etc.), a credit card accountsubject to prior fraudulent activity, or a credit card account revokedby the financial institution. Further, the one or more filtrationcriteria may cause executed training input module 166 to performoperations that exclude, from first subset 168A and second subset 168B,a consolidated data record of any customer associated with a personal orbusiness bankruptcy, or any deceased customer. The disclosed embodimentsare not limited to these exemplary filtration criteria, and in otherinstances, executed training input module 166 may also performoperations that filter the consolidated data records of first subset168A and second subset 168B in accordance with any additional, oralternate, filtration criteria appropriate to the consolidated oraggregated elements of customer profile, account, delinquency,credit-bureau, and/or transaction data.

Referring back to FIG. 1B, executed training input module 166 mayperform operations that generate a plurality of training datasets 170based on elements of data obtained, extracted, or derived from all or aselected portion of first subset 168A of the consolidated data records.In some instances, the plurality of training datasets 170 may, whenprovisioned to an input layer of the gradient-boosted decision-treeprocess described herein, enable executed training engine 162 to trainthe gradient-boosted decision-tree process to predict, during a currenttemporal interval, a likelihood of occurrences of default eventsinvolving customers of the financial institution during a futuretemporal interval.

By way of example, each of the plurality of training datasets 170 may beassociated with a corresponding one of the customers of the financialinstitution and a corresponding temporal interval, and may include,among other things a customer identifier associated with thatcorresponding customer and a temporal identifier representative of thecorresponding temporal interval, as described herein. Each of theplurality of training datasets 170 may also include elements of data(e.g., feature values) that characterize the corresponding one of thecustomers, the corresponding customer's interaction with the financialinstitution or with another financial institution, and/or an occurrence(or lack thereof) of default events involving the corresponding customerduring a temporal interval disposed prior to the corresponding temporalinterval, e.g., the extraction interval Δt_(extract) described herein.

For instance, plurality of training datasets 170 may include a value ofone or more numerical input features, and examples of the numericalinput features include, but are not limited to, a customer age, anoutstanding balance associated with a credit products, such as thecredit-card account described herein, a past-due balance or a past-dueinterval associated with the credit-card account, a time-averaged valueof transactions involving the credit-card account, or a time-averagevalue of deposits into a corresponding deposit account. Additionally, insome instances, the plurality of training datasets 170 may include avalue one or more categorical input features, and examples of thecategorical input features include, but are not limited to, a customertype (e.g., personal banking, small business banking), a type ofcredit-card account held by a customer (e.g., a secured credit-cardaccount, a rewards-based credit card account), or a type of demandaccount held by the customer (e.g., high-yield checking, etc.). Further,and as described herein, each of training datasets 170 may also includean element of ground-truth data indicative of the presence or absence ofa default event associated with a corresponding one of the customerswithin a temporal period, such as a twelve-month period, subsequent tothe corresponding temporal interval (e.g., as specified by thecorresponding temporal identifier).

In some instances, executed training input module 166 may performoperations that identify, and obtain or extract, one or more of thefeatures values from the consolidated data records maintained withinfirst subset 168A and associated with the corresponding one of thecustomers. The obtained or extracted feature values may, for example,include elements of the customer profile, account, delinquency,credit-bureau, or transaction data described herein (e.g., which maypopulate the consolidated data records maintained within first subset168A). The disclosed embodiments are, however, not limited to theseexamples of obtained or extracted feature values, and in otherinstances, training datasets 170 may include any additional or alternateelement of data extracted or obtained from the consolidated data recordsof first subset 168A, associated with corresponding one of thecustomers, and associated with the extraction interval Δt_(extract)described herein.

Further, in some instances, executed training input module 166 mayperform operations that compute, determine, or derive one or more of thefeatures values based on elements of data extracted or obtained from theconsolidated data records maintained within first subset 168A. Examplesof these computed, determined, or derived feature values may include,but are not limited to, time-average values of payments associated withone or more financial products or payment instruments held bycorresponding ones of the customer, time-average balances associatedwith these financial products, sums of balances associated with variousfinancial products or payment instruments held by corresponding ones ofthe customers, total amounts of credit available to corresponding onesof the customers, and/or total numbers of past-due balances ordelinquencies associated with corresponding ones of the customers. Thesedisclosed embodiments are, however, not limited to these examples ofcomputed, determined, or derived feature values, and in other instances,training datasets 170 may include any additional or alternate featuredcomputed, determine, or derived from data extracted or obtained from theconsolidated data records of first subset 168A, associated withcorresponding one of the customers, and associated with the extractioninterval Δt_(extract) described herein.

Referring back to FIG. 1B, executed training input module 166 mayprovide training datasets 170 as an input to an adaptive training andvalidation module 172 of executed training engine 162. In someinstances, and upon execution by the one or more processors of FIcomputing system 130, adaptive training and validation module 172 mayperform operations that establish a plurality of nodes and a pluralityof decision trees for the gradient-boosted, decision-tree process, withmay ingest and process the elements of training data (e.g., the customeridentifiers, the temporal identifiers, the feature values, etc.)maintained within each of the plurality of training datasets 170.Further, and based on the execution of adaptive training and validationmodule 172, and on the ingestion of each of training datasets 170 by theestablished nodes of the gradient-boosted, decision-tree process, FIcomputing system 130 may perform operations that train thegradient-boosted, decision-tree process against the elements of trainingdata included within each of training datasets 170.

In some examples, the distributed components of FI computing system 130may execute adaptive training and validation module 172, and may performany of the exemplary processes described herein in parallel to train thegradient-boosted, decision-tree process against the elements of trainingdata included within each of training datasets 170. The parallelimplementation of adaptive training and validation module 172 by thedistributed components of FI computing system 130 may, in someinstances, be based on an implementation, across the distributedcomponents, of one or more of the parallelized, fault-tolerantdistributed computing and analytical protocols described herein.

Through the performance of these exemplary training processes, executedadaptive training and validation module 172 may perform operations thatcompute one or more candidate process parameters that characterize thetrained, gradient-boosted, decision-tree process, and package thecandidate process parameters into corresponding portions of candidateprocess data 173A. In some instances, the candidate process parametersincluded within candidate process data 173A may include, but are notlimited to, a learning rate associated with the trained,gradient-boosted, decision-tree process, a number of discrete decisiontrees included within the trained, gradient-boosted, decision-treeprocess (e.g., the “n_estimator” for the trained, gradient-boosted,decision-tree process), a tree depth characterizing a depth of each ofthe discrete decision trees included within the trained,gradient-boosted, decision-tree process, a minimum number ofobservations in terminal nodes of the decision trees, and/or values ofone or more hyperparameters that reduce potential process overfitting(e.g., regularization of pseudo-regularization hyperparameters).Further, and based on the performance of these exemplary trainingprocesses, executed adaptive training and validation module 172 may alsogenerate candidate input data 173B, which specifies a candidatecomposition of an input dataset for the trained, gradient-boosted,decision-tree process (e.g., which be provisioned as inputs to the nodesof the decision trees of the trained, gradient-boosted, decision-treeprocess).

As illustrated in FIG. 1B, executed adaptive training and validationmodule 172 may provide candidate process data 173A and candidate inputdata 173B as inputs to executed training input module 166 of trainingengine 162, which may perform any of them exemplary processes describedherein to generate a plurality of validation datasets 174 havingcompositions consistent with candidate input data 173B. As describedherein, the plurality of validation datasets 174 may, when provisionedto, and ingested by, the nodes of the decision trees of the trained,gradient-boosted, decision-tree process, enable executed training engine162 to validate the predictive capability and accuracy of the trained,gradient-boosted, decision-tree process, for example, based on elementsof ground truth data incorporated within the validation datasets 174, orbased on one or more computed metrics, such as, but not limited to,computed precision values, computed recall values, and computed areaunder curve (AUC) for receiver operating characteristic (ROC) curves orprecision-recall (PR) curves.

By way of example, executed training input module 166 may parsecandidate input data 173B to obtain the candidate composition of theinput dataset, which not only identifies the candidate elements ofcustomer-specific data included within each validation dataset (e.g.,the candidate feature values described herein), but also a candidatesequence or position of these elements of customer-specific data withinthe validation dataset. Examples of these candidate feature valuesinclude, but are not limited to, one or more of the feature valuesextracted, obtained, computed, determined, or derived by executedtraining input module 166 and packaged into corresponding portions oftraining datasets 170, as described herein. For instance, the candidatefeature values may include one or more of the feature values extracted,obtained, computed, determined, or derived from elements of the customeraccount, account, or delinquency data described herein, either alone orin conjunction with one or more additional feature values extracted,obtained, computed, determined, or derived from the elements ofcredit-bureau data described herein.

Further, in some examples, each of the plurality of validation datasets174 may be associated with a corresponding one of the customers of thefinancial institution, and with a corresponding temporal interval withinthe validation interval Δt_(validation), and executed training inputmodule 166 may access the consolidated data records maintained withinsecond subset 168B of consolidated data store 144, and may performoperations that extract, from an initial one of the consolidated datarecords, a customer identifier (which identifies a corresponding one ofthe customers of the financial institution associated with the initialone of the consolidated data records) and a temporal identifier (whichidentifies a temporal interval associated with the initial one of theconsolidated data records). Executed training input module 166 maypackage the extracted customer identifier and temporal identifier intoportions of a corresponding one of validation datasets 174, e.g., inaccordance with candidate input data 173B.

Executed training input module 166 may perform operations that accessone or more additional ones of the consolidated data records that areassociated with the corresponding one of the customers (e.g., thatinclude the customer identifier) and as associated with a temporalinterval (e.g., based on corresponding temporal identifiers) disposedprior to the corresponding temporal interval, e.g., within theextraction interval Δt_(extract) described herein. Based on portions ofcandidate input data 173B, executed training input module 166 mayidentify, and obtain or extract, one or more of the feature values ofthe validation datasets from within the additional ones of theconsolidated data records within second subset 168B. Further, in someexamples, and based on portions of candidate input data 173B, executedtraining input module 166 may perform operations that compute,determine, or derive one or more of the features values based onelements of data extracted or obtained from further ones of theconsolidated data records within second subset 168B. Executed traininginput module 166 may package each of the obtained, extracted, computed,determined, or derived feature values into corresponding positionswithin the initial one of validation datasets 174, e.g., in accordancewith the candidate sequence or position specified within candidate inputdata 173B. Additionally, and in some examples, executed training inputmodule 166 may also package, into an appropriate position within aportion of the corresponding one of validation datasets 174, an elementof ground-truth data indicative of the presence or absence of a defaultevent associated with the corresponding one of the customers within atemporal period, such as a twelve-month period disposed subsequent tothe corresponding temporal interval.

In some instances, executed training input module 166 may perform any ofthe exemplary processes described herein to generate additional, oralternate, ones of validation datasets 174 based on the elements of datamaintained within the consolidated data records of second subset 168B.For example, each of the additional, or alternate, ones of validationdatasets 174 may be, associated with a corresponding, and distinct, pairof customer and temporal identifiers, and as such, correspondingcustomers of the financial institution and corresponding temporalintervals within validation interval Δt_(validation). Further, executedtraining input module 166 may perform any of the exemplary processesdescribed herein to generate an additional, or alternate, ones ofvalidation datasets 174 associated with each unique pair of customer andtemporal identifiers maintained within the consolidated data records ofsecond subset 168B, and in other instances a number of discretevalidation datasets within validation datasets 174 may be predeterminedor specified within candidate input data 173B.

Referring back to FIG. 1B, executed training input module 166 mayprovide the plurality of validation datasets 174 as inputs to executedadaptive training and validation module 172. In some examples, executedadaptive training and validation module 172 may perform operations thatapply the trained, gradient-boosted, decision-tree process to respectiveones of validation datasets 174 (e.g., based on the candidate processparameters within candidate process data 173A, as described herein), andthat generate elements of validation output data 176 based on theapplication of the trained, gradient-boosted, decision-tree process tocorresponding ones of validation datasets 174.

As described herein, each of the each of elements of validation outputdata 176 may be generated through the application of the trained,gradient-boosted, decision-tree process to a corresponding one ofvalidation datasets 174, which may include, among other things, acustomer identifier (e.g., identifying a corresponding customer of thefinancial institution), a temporal identifier (e.g., identifying acorresponding temporal interval), and an element of ground-truth data,which indicates whether the corresponding customer is involved in anactual default event during a future temporal interval, e.g., the targetinterval Δt_(target). Further, as described herein, each of elements ofvalidation output data 176 may be representative of a predictedlikelihood of an occurrence of a default event involving, or associatedwith, the corresponding customer during the target interval Δt_(target),and in some instances, the predicted likelihood may be represented by anumerical score of either zero (e.g., indicative of a predictednon-occurrence of the default event during the target intervalΔt_(target)) or unity (e.g., indicative of a predicted occurrence of thedefault event during the target interval Δt_(target)).

Executed adaptive training and validation module 172 may performoperations that compute a value of one or more metrics that characterizea predictive capability, and an accuracy, of the trained,gradient-boosted, decision-tree process based on the generated elementsof validation output data 176 and corresponding ones of validationdatasets 174. The computed metrics may include, but are not limited to,one or more recall-based values for the trained, gradient-boosted,decision-tree process (e.g., “recall@5,” “recall@10,” “recall@20,”etc.), and additionally, or alternatively, one or more precision-basedvalues for the trained, gradient-boosted, decision-tree process.Further, in some examples, the computed metrics may include a computedvalue of an area under curve (AUC) for a precision-recall (PR) curveassociated with the trained, gradient-boosted, decision-tree process,and additional, or alternatively, computed value of an AUC for areceiver operating characteristic (ROC) curve associated with thetrained, gradient-boosted, decision-tree process. The disclosedembodiments are, however, not limited to these exemplary computed metricvalues, and in other instances, executed adaptive training andvalidation module 172 may compute a value of any additional, oralternate, metric appropriate to validation output data 176, validationdatasets 174, the elements of ground-truth data, or the trained,gradient-boosted, decision-tree process.

In some examples, executed adaptive training and validation module 172may also perform operations that determine whether all, or a selectedportion, of the computed metric values satisfy one or more thresholdconditions for a deployment of the trained, gradient-boosted,decision-tree process and a real-time application to elements ofcustomer profile, account, transaction, delinquency, or credit-bureaudata, as described herein. For instance, the one or more thresholdconditions may specify one or more predetermined threshold values forthe trained, gradient-boosted, decision-tree process, such as, but notlimited to, a predetermined threshold value for the computedrecall-based values, a predetermined threshold value for the computedprecision-based values, and/or a predetermined threshold value for thecomputed AUC values. In some examples, executed adaptive training andvalidation module 172 performs operations that establish whether one, ormore, of the computed recall-based values, the computed precision-basedvalues, or the computed AUC values exceed, or fall below, acorresponding one of the predetermined threshold values and as such,whether the trained, gradient-boosted, decision-tree process satisfiesthe one or more threshold requirements for deployment.

If, for example, executed adaptive training and validation module 172were to establish that one, or more, of the computed metric values failto satisfy at least one of the threshold requirements, FI computingsystem 130 may establish that the trained, gradient-boosted,decision-tree process is insufficiently accurate for deployment and areal-time application to the elements of customer profile, account,transaction, delinquency, and/or credit-bureau data described herein.Executed adaptive training and validation module 172 may performoperations (not illustrated in FIG. 1B) that transmit data indicative ofthe established inaccuracy to executed training input module 166, whichmay perform any of the exemplary processes described herein to generateone or more additional training datasets and to provision thoseadditional training datasets to executed adaptive training andvalidation module 172. In some instances, executed adaptive training andvalidation module 172 may receive the additional training datasets, andmay perform any of the exemplary processes described herein to trainfurther the gradient-boosted, decision-tree process against the elementsof training data included within each of the additional trainingdatasets.

Alternatively, if executed adaptive training and validation module 172were to establish that each computed metric value satisfies thresholdrequirements, FI computing system 130 may deem the gradient-boosted,decision-tree process trained, and ready for deployment and real-timeapplication to the elements of customer profile, account, transaction,delinquency, and/or credit-bureau data described herein. In someinstances, executed adaptive training and validation module 172 maygenerate process parameter data 175A that includes the processparameters of the trained, gradient-boosted, decision-tree process, suchas, but not limited to, each of the candidate process parametersspecified within candidate process data 173A. Further, executed adaptivetraining and validation module 172 may also generate process input data175B, which characterizes a composition of an input dataset for thetrained, gradient-boosted, decision-tree process and identifies each ofthe discrete data elements within the input data set, along with asequence or position of these elements within the input data set (e.g.,as specified within candidate input data 173B). As illustrated in FIG.1B, executed adaptive training and validation module 172 may performoperations that store process parameter data 175A and process input data175B within the one or more tangible, non-transitory memories of FIcomputing system 130, such as consolidated data store 144.

B. Exemplary Techniques for Training Explainability Processes Associatedwith Trained, Gradient-Boosted Decision-Tree Processes within aDistributed Computing Environment

In some examples, one or more of the distributed components of FIcomputing system 130 may perform operations, described herein, thatadaptively train a machine learning or artificial intelligence processto predict, during a current temporal interval, a likelihood of anoccurrence of an event, such as one or more of the exemplary defaultevents described herein, during a future temporal interval usingtraining data associated with a first prior temporal interval, and usingvalidation data associated with a second, and distinct, prior temporalinterval. As described herein, the machine-learning orartificial-intelligence process may include an ensemble or decision-treeprocess, such as a gradient-boosted, decision-tree process, and uponcompletion of the training and validation processes described herein,the one or more distributed components of FI computing system 130 mayperform any of the exemplary processes described herein to generateelements of process parameter data that includes the process parametersof the trained, gradient-boosted, decision-tree process, such as, butnot limited to, the exemplary process parameters described herein (e.g.,process parameter data 175A of FIG. 1B), and to generate elements ofprocess input data that characterizes a structure and a composition of acustomer-specific input dataset for the trained, gradient-boosted,decision-tree process (e.g., process input data 175B of FIG. 1B).

The elements of process input data may identify each of the numerical orcategorical input features included within the customer-specific inputdataset, and along with a sequence or position of a value of each of thenumerical or categorical input features within the customer-specificinput dataset. FI computing system may perform operations that store theelements of process parameter data and process input data within a datarepository, such as consolidated data store 144, in conjunction withall, or a subset of the validation datasets, which may be structured inaccordance with the elements of process input data (e.g., validationdatasets 174 of FIG. 1B), and corresponding elements of output datagenerated through the application of the trained, gradient-boosteddecision tree process to corresponding ones of the validation datasets(e.g., validation output data 176 of FIG. 1B).

Further, the one or more distributed components of FI computing system130 may also perform operations, described herein, that train anexplainability process against elements of one or more validationdatasets associated with the trained, gradient-boosted, decision-treeprocess, such as, but not limited to, one or more of validation datasets174B of FIG. 1B. As described herein, each of validation datasets 174Bmay include values of numerical or categorical input features specifiedby the elements of process input data 175B, and the explainabilityprocess may include, but us not limited to, a Shapley splitter processthat leverages Shapley additive explanations to decompose output datagenerated through an application of the trained gradient-boosted,decision-tree processes to a corresponding input dataset, and tocharacterize an impact of a value of each of the numerical orcategorical input features on the predicted output, e.g., based on amagnitude of a corresponding, input-feature-specific Shapley value.

By way of example, and for a particular input feature, a Shapley valueof large magnitude may imply that a value of the particular inputfeature is associated with a corresponding, large contribution to thepredicted output, which may drive an increase a magnitude of thatpredicted output. Further, for a particular input feature, a Shapleyvalue of small magnitude may imply that a value of the particular inputfeature is associated with a corresponding, small contribution to thepredicted output and to any increase in the magnitude of that predictedoutput. Further, and in view of these relationships, if a value of theparticular input feature were to exceed a determined, threshold featurevalue, then a Shapley value associated with the particular input featurevalue would be likely to exceed a corresponding threshold Shapley value,which may indicate that any increase in the value of the particularinput feature would also drive an increase in the predicted output ofthe trained, gradient-boosted decision-tree process (e.g., a valueindicative of a predicted likelihood of an occurrence of a default eventinvolving a customer of the financial institution and a correspondingcredit-card account during the future temporal interval, as describedherein).

The input dataset may, for instance, include values of one or morenumerical input features, and the one or more distributed computingcomponents of FI computing system 130 may perform any of the exemplaryprocesses described herein to train the Shapley splitter process againstthe elements of one or more of validation datasets 174 and generate, foreach of the numerical input features (e.g., as specified by the elementsof process input data 175), a threshold feature value v* and a thresholdShapley value s*. Further, and as described herein, when a value of aparticular numerical input feature v within validation datasets 174exceeds the corresponding threshold feature value (e.g., v≥v*), theresulting Shapley value s would be likely to exceed the correspondingthreshold Shapley value (e.g., s≥s*).

By way of example, and for a particular one of the numerical inputfeature, the one or more distributed computing components of FIcomputing system 130 may perform operations that associate each of thefeature value of the particular numerical input feature (e.g., asincluded within a plurality N of validation datasets) with acorresponding Shapley feature value, and generate a correspondingplurality of pairs x_(i) of associated feature values v_(i) and Shapleyfeature values s_(i) (e.g., x_(i)|_(i=1) ^(N){(v_(i),s_(i))}_(i=1)^(N)). FIG. 2A illustrates an exemplary Shapley scatter plot 200 of thecorresponding plurality of pairs x_(i) of associated feature valuesv_(i) and Shapley feature values s_(i) for the particular numericalinput feature. As illustrated in FIG. 2A, Shapley scatter plot 200identifies values of the particular numerical input feature along axis202A, and corresponding Shapley values along axis 202B, and each of thediscrete points within Shapley scatter plot 200 may corresponding to adiscrete one of the pairs x_(i) of associated feature values v_(i) andShapley feature values s_(i). Further, in some instances, Shapleyscatter plot 200 may be decomposed into four discrete regions, showngenerally as regions 204A, 204B, 204C, and 204D, by lines 206A and 206Brepresentative, respectively, a threshold value v* and a thresholdShapley feature value s* for the particular numerical input feature.

For example, region 204A may include those validation instancescharacterized by feature values v_(i) that fail to exceed the thresholdfeature value v* and Shapley values s_(i) that exceed the thresholdShapley value s*, and region 204B may include those validation instancescharacterized by feature values v_(i) that exceed the threshold featurevalue v* and Shapley values s_(i) that exceed the threshold Shapleyvalue s*. Further, as illustrated in FIG. 2A, region 204C may includethose validation instances characterized by feature values v_(i) thatfail to exceed the threshold feature value v* and Shapley values s_(i)that fail to exceed the threshold Shapley value s*, and region 204D mayinclude those validation instances characterized by feature values v_(i)that exceed the threshold feature value v* and Shapley values s_(i) thatfail to exceed the threshold Shapley value s*. By way of example,regions 204A, 204B, 204C, and 204D may collectively establish aconfusion matrix, with regions 204B and 204D (e.g., including thevalidation instances characterized by feature values v_(i) that exceedthe threshold feature value v*) corresponding to “predicted positive”sample, and with regions 204A and 204B (e.g., including the validationinstances characterized by Shapley values s_(i) that exceed thethreshold Shapley value s) corresponding to “ground-truth” positivesamples.

Further, a number of discrete validation instances disposed withinrespective ones of regions 204A, 204B, and 204D of Shapley scatter plot200 may facilitate a computation of corresponding ones of a precisionvalue and a recall value for the particular numerical input featureassociated with Shapley scatter plot 200. For example, the precisionvalue may be defined as B/B+D, and the recall value may be defined asB/A+B, where A corresponds to the number of discrete validationinstances disposed within region 204A of Shapley scatter plot 200, Bcorresponds to the number of discrete validation instances disposedwithin region 204B of Shapley scatter plot 200, and D corresponds to thenumber of discrete validation instances disposed within region 204D ofShapley scatter plot 200. Additionally, an F₁ score for the particularnumerical input feature associated with Shapley splitter plot 200 may bedefined as a harmonic mean of the recall value and the precision value,and may be expressed as 2B/2B+A+D. In some instances, illustrated inFIG. 2B, the one or more distributed components of FI computing system130 may perform any of the exemplary processes described herein todetermine a threshold feature value v* and a threshold Shapley value s*for the particular numerical input feature that minimizes a number ofthe corresponding validation instances x; that are disposed withinregions 204A and 204D the Shapley scatter plot 200, and as such, thatmaximize a corresponding F₁ score for the particular numerical inputfeature associated with Shapley splitter plot 200.

As described herein, when applied to a customer-specific input datasetthat includes a corresponding value of the particular numerical inputfeature by the one or more distributed components of FI computing system130, the trained Shapley splitter process may generate elements oftextual content that associate a magnitude of the corresponding valuewith the output data generated through the application of the trainedgradient-boosted, decision-tree process to the customer-specific inputdataset and as such, with an adverse decision associated with the outputdata (e.g., that the corresponding numerical feature value is “too highor too low”). Further, and as described herein, the one or moredistributed components of FI computing system 130 may also perform anyof these exemplary processes to train further the Shapley splitterprocess against the elements of one or more of validation datasets 174Band generate a corresponding threshold feature value v* and acorresponding threshold Shapley value s* for each additional, oralternate, ones of the numerical input features specified within theelements of process input data 175B.

The input dataset may also include, among other things, values of one ormore categorical input features, and as illustrated in FIG. 2B, the oneor more distributed components of FI computing system 130 may alsoperform any of the exemplary described herein to train the Shapleysplitter process against the elements of one or more of validationdatasets 174B and to determine a threshold Shapley value s*for each ofthe categorical input features. When applied to a customer-specificinput dataset that includes a corresponding value of a particularcategorical input feature by the one or more distributed components ofFI computing system 130, the trained Shapley splitter process maygenerate elements of textual content that associate a category specifiedby the corresponding value, and a Shapley value that exceeds thecorresponding threshold Shapley value s*, with the output data generatedthrough the application of the trained gradient-boosted, decision-treeprocess to the customer-specific input dataset and as such, with anadverse decision associated with the output data (e.g., that theparticular categorical feature value “does or does not belong to acategory”).

Referring to FIG. 2B, an explainability engine 210 executed by the oneor more processors of FI computing system 130 (e.g., by one, or more,distributed components of FI computing system 130, described herein),may perform operations that access validation datasets 174 andcorresponding elements of validation output data 176 maintained withinconsolidated data store 144. As described herein, each of the elementsof validation output data 176 may be associated with a corresponding oneof validation datasets 174 and may be generated through an applicationof the trained, gradient-boosted, decision-tree process to thecorresponding one of validation datasets 174. Further, each of obtainedvalidation datasets 174 may be structured in accordance with theelements of process input data 175B, and may include values of aplurality of input features identified and specified by the elements ofprocess input data 175B, such as the numerical or categorical inputfeatures described herein. In some instances, executed explainabilityengine 210 may perform any of the exemplary processes described hereinto compute, based on validation datasets 174 and the elements of outputdata 176, a feature value contribution, such as a Shapley feature valuecontribution, for each of the plurality of input features thatcharacterizes a contribution of the corresponding input feature on theoutcome of the trained, gradient-boosted, decision-tree process, suchas, but not limited to, the predicted likelihood of an occurrence of adefault involving a corresponding customer and a credit-card accountduring a future temporal interval.

By way of example, executed explainability engine 210 may performoperations that, based on one or more of validation datasets 174,generate a plurality of modified validation datasets 212 associated withcorresponding ones of the input features specified within the elementsof process input data 175B, and that provision each of modifiedvalidation datasets 212 as an input to a predictive engine 214 executedby the one or more processors of FI computing system 130 (e.g., based ona programmatic signal generated by executed explainability engine 210,etc.). For instance, and for a numerical input feature identified withinprocess input data 175B, executed explainability engine 210 maydetermine a range of the corresponding input feature values includedwithin validation datasets 174, and may perform operations thatdiscretize the determined range into discrete intervals (e.g.,consistent with a predetermined number of interpolation points, etc.)and that compute, for each of the discrete intervals, a discretizedfeature value. By way of example, the discretized feature values mayvary linearly across the discretized intervals of the feature range, orin accordance with any additional, or alternate non-linear or linearfunction.

Executed explainability engine 210 may perform operations that packagethe discretized feature values into a corresponding set of discretizedfeature values for the numerical input feature, and that generate, forthe numerical input feature, a subset of modified validation datasets212 based on a perturbation of one, or more, of validation datasets 174based on the corresponding set of discretized feature values. By way ofexample, and for corresponding one of validation datasets 174 and thenumerical input feature, executed explainability engine 210 may performany of the exemplary processes described herein to identify, within thecorresponding one of validation datasets 174, the input feature valueassociated with the numerical input feature, and to generatecorresponding ones of modified validation datasets 212 by replacing thatfeature value with a corresponding one of the discretized feature valuesfor the numerical input feature.

Further, in some instances, and for a categorical input featureidentified within process input data 175B, executed explainabilityengine 210 may identify each of the discrete feature values (e.g.,distinct categories) associated with the categorical input feature, andmay perform operations that generate, for the categorical input feature,a subset of modified validation datasets 212 based on the correspondingones of the discrete feature values. By way of example, and forcorresponding one of validation datasets 174 and the categorical inputfeature, executed explainability engine 210 may perform any of theexemplary processes described herein to identify, within thecorresponding one of validation datasets 174, the input feature valueassociated with the categorical input feature, and to generatecorresponding ones of modified validation datasets 212 by replacing thatfeature value with a corresponding one of the discretized feature valuesfor the categorical input feature (the distinct categories, including,in some instances, a null value). The disclosed embodiments are,however, not limited to these exemplary processes, and in otherinstances, executed explainability engine 210 generate subsets ofmodified validation datasets 212 for corresponding ones of the numericalor categorical features using any additional, or alternate, processappropriate to the categorical or numerical features or to the featurevalues maintained within validation datasets.

Executed explainability engine 210 may also perform one or more of theexemplary processes described herein to generate a corresponding subsetof modified validation datasets 212 for each additional, or alternate,one of the numerical or categorical features specified by the elementsof process input data 175B, and executed explainability engine 210 mayprovision each of modified validation datasets 212 as input to executedpredictive engine 214. In some instances, illustrated in FIG. 2B,executed predictive engine 214 may perform operations that obtain, fromconsolidated data store 144, process parameter data 175A that includesone or more process parameters of the trained, gradient-boosted,decision-tree process, and based on portions of process parameter data175A, executed predictive engine 214 may perform operations thatestablish a plurality of nodes and a plurality of decision trees for thetrained, gradient-boosted, decision-tree process, each of which receive,as inputs (e.g., “ingest”), corresponding elements of modifiedvalidation datasets 212. Based on the execution of predictive engine214, and based on the ingestion of modified validation datasets 212 bythe established nodes and decision trees of the trained,gradient-boosted, decision-tree process, FI computing system 130 mayperform operations that apply the trained, gradient-boosted,decision-tree process to each of modified validation datasets 212, andthat generate an element of predicted output data 216 associated with acorresponding one of modified validation datasets 212.

Based on elements of predicted output data 216, executed explainabilityengine 210 may perform any of the exemplary processes described hereinto generate one or more elements of explainability data 218 thatcharacterize, among other things, a marginal effect of a perturbation ina value of each of the input features specified within process inputdata 175B on an outcome of the trained, gradient-boosted, decision-treeprocess, and a contribution of each of the input features (e.g., thenumerical or categorical features described herein) to the predictedoutput data generated by an application of the trained,gradient-boosted, decision-tree process to customer-specific inputdatasets (e.g., the predicted likelihood that the corresponding one ofthe customers will be involved in a default event associated with acredit-card account during the future temporal interval, etc.). By wayof example, and as described herein, executed explainability engine 210may compute a Shapley value feature for each of the input features basedon the elements of validation output data 176, the elements of predictedoutput data 216, and additionally, or alternatively, corresponding onesof modified validation datasets 212. In some instances, executedexplainability engine 210 may calculate the Shapley feature values inaccordance with a Shapley Additive exPlanations (SHAP) algorithm (e.g.,when the selected machine learning or artificial intelligence processcorresponds to a gradient-boosted decision tree algorithm), or inaccordance with an integrated gradient algorithm (e.g., when theselected machine learning or artificial intelligence process correspondsto a deep neural-network models).

Executed explainability engine 210 may perform operations that packageeach of the Shapley value features into a corresponding portion ofexplainability data 218, wither alone or in conjunction withcorresponding feature values maintained within validation datasets 174.In some instances, executed explainability engine 210 may performoperations that extract, from one or more of validation datasets 174,values of each of the input features, and may generate a plurality ofelements of sampling data 220, each of which includes a training samplethat associate a corresponding one of the extracted values of the inputfeatures, with a corresponding feature identifier (e.g., as obtainedfrom the elements of process input data 175B) and with a correspondingone of the computed Shapley feature values. By way of example, thetraining sample included within element 222A may include a featureidentifier 224 of a numerical or categorical feature specified withinelements of process input data 175B (e.g., an alphanumeric characterstring associated with, or assigned by, FI computing system 130, etc.),a corresponding value 226 of a numerical or categorical feature (e.g.,feature value v_(i)), and a corresponding Shapley feature value 228associated with that numerical or categorical feature (e.g., Shapleyfeature value s_(i)). In some instances, executed explainability engine210 may provide explainability data 218, including the discrete elementsof sampling data 220 as an input to a training engine 230 that, uponexecution by one or more processors of FI computing system 130, train aShapley-splitter process against the training samples maintained withinthe elements of sampling data 220 to generate, for each of the inputfeatures (e.g., as specified by the elements of process input data175B), a threshold feature value v* (or a threshold category c*) and athreshold Shapley value s*.

As described herein, the threshold feature value v* associated with anumerical feature may include a numerical value, and the thresholdfeature value v* associated with a categorical feature may include analphanumeric character string identifying a corresponding category.Further, when applied to a customer-specific input dataset (e.g., thatincludes values of the input features specified within process inputdata 175B), the trained Shapley splitter process may leverage arelationship between the feature values of customer-specific inputdataset and the corresponding Shapley feature values, whichcharacterizes a contribution of the at least one feature value to thepredicted output of the trained gradient-boosted, decision-tree process,and generate elements of textual content that characterize anassociation between the one or more of the feature values and thepredicted output (e.g., a feature value is “too high or too low,” or afeature value “does or does not belong to a category,” etc.).

Referring back to FIG. 2B, training engine 230 may receiveexplainability data 218, including the elements of sampling data 220,from executed explainability engine 210, and a numerical-featuretraining module 232 of executed training engine 230 parse the featureidentifiers maintained within the elements of sampling data 220 toobtain a plurality of the training samples associated with each, or atargeted subset of, the numerical input features specified by theelements of process input data 175B. In some instances, executednumerical-feature training module 232 may perform operations that sortobtained training samples into feature-specific subsets associated withcorresponding ones of the numerical input features, and further, thatsort the training samples within each of the feature-specific subsets inaccordance with the Shapley feature values (e.g., in descending orderbased on the corresponding Shapley feature values, etc.). In someinstances, executed numerical-feature training module 232 may select oneof the numerical input feature for training (e.g., numerical feature fassociated with a corresponding feature identifier f_(ID)) and mayobtain the sorted training samples maintained within thefeature-specific subset associated with the selected numerical inputfeature (e.g., a plurality of N training samples{(f_(ID),v_(i),s_(i))}_(i=1) ^(N), where v_(i) corresponds to thefeature value of the selected numerical input feature within the i^(th)training sample, and s_(i) corresponds to the Shapley feature value ofthe selected numerical input feature within the i^(th) training sample).

By way of example, the numerical feature values v_(i) and associatedShapley feature values s_(i) maintained with the sorted training samplesmay establish a corresponding Shapley scatter plot associated selectednumerical input feature, such as Shapley scatter plot 200 of FIG. 2A,and each of the N pairs of the numerical feature values v_(i) andassociated Shapley feature values s_(i) may represent a corresponding“point” within the Shapley scatter plot. In some instances, in trainingthe Shapley splitter process, executed numerical-feature training module232 may perform operations that determine the threshold feature value v*and the threshold Shapley value s* for the selected numerical inputfeature that maximize the values of precision and recall for theselected numerical input feature and as such, that maximize the F₁ scoreassociated with the selected numerical feature. By way of example, andthrough an implementation of one or more of the exemplary trainingprocesses described herein, executed numerical-feature training module232 the threshold feature value v* and the threshold Shapley value s*for the selected numerical input feature in accordance with:

${\left( {v^{*},s^{*}} \right) = {\underset{{({v,s})} \in R^{2}}{\arg\max}\frac{2{B\left( {v,s} \right)}}{{2{B\left( {v,s} \right)}} + {A\left( {v,s} \right)} + {D\left( {v,s} \right)}}}},$

where: A(v,s) corresponds to a number of the sorted training sampleshaving a numerical feature value v_(i) that fails to exceed thethreshold feature value v* and a Shapley feature value s_(i) thatexceeds the threshold Shapley value s*; B(v,s) corresponds to a numberof the sorted training samples having a numerical feature value v_(i)that exceeds the threshold feature value v* and a Shapley feature values_(i) that exceeds the threshold Shapley value s*; and D(v,s)corresponds to a number of the sorted training samples having anumerical feature values v_(i) that exceed the threshold feature valuev* and a Shapley feature value s_(i) that fails exceeds the thresholdShapley value s*.

By way of example, executed numerical-feature training module 232 mayperform operations that assign each the Shapley feature values s_(i)maintained within the sorted training samples to a corresponding one ofa predetermined number n_(bin) of Shapley-value bins (e.g., that “bin”the Shapley feature values into the predetermined number ofShapley-value bins), and based on the binned Shapley feature values,determine a plurality of candidate Shapley threshold values for theselected numerical input feature (e.g., s^((j)), where j=1, . . . ,n_(bin)). In some instances, the Shapley feature values s_(i) maintainedwithin the sorted, training samples of the feature-specific subsetassociated with the selected numerical input feature may include, andmay be bounded by, a maximum Shapley feature value s_(MAX). Executednumerical-feature training module 232 may also establish apredetermined, minimum value s_(min) for the threshold Shapley values*(e.g., such that s*≥s_(min)), and may establish a predetermined,maximum percentage p_(min) of the sorted, training samples that arecharacterized by Shapley feature values s_(i) that exceed the thresholdShapley value s*. In some instances, executed numerical-feature trainingmodule 232 may perform operations that compute the plurality ofcandidate Shapley threshold values s^((j)) for the selected numericalinput feature across a range of Shapley feature values having an upperbound defined by the maximum Shapley feature value s_(MAX), and a lowerbound s_(m) defined by a maximum of the predetermined, minimum values_(min), or by a corresponding one of the Shapley feature values s_(i)associated within the predetermined, maximum percentage p_(min) of thesorted, training samples, e.g., in accordance withs^((j))=(1−t_(j))s_(m)+t_(j) s_(MAX), where t_(j)=j/n_(bin), and wherej=1, . . . , n_(bin).

Further, and for each of the j candidate Shapley threshold valuess^((j)), executed numerical-feature training module 232 may also: (i)establish F₁(i,j) as the F₁ score computing using s*=s^((j)) and v*beingequivalent to a corresponding one of the numerical feature values havingthe i^(th) largest magnitude; and (ii) establish {circumflex over(F)}₁(i,j) as the F₁ score computing using s*=s^((j)) and v*beingequivalent to a corresponding one of the numerical feature values havingthe i^(th) smallest magnitude. In some instances, executednumerical-feature training module 232 may also perform operations thatdetermine the integer values of index i (e.g., ranging from unity to N)and index j (e.g., ranging from unity to n_(bin)) resulting in a maximumvalue of F₁(i,j) or alternatively, a maximum value of {circumflex over(F)}₁(i,j). In some instances, the operations performed by executednumerical-feature training module 232, which determine the integervalues of indices i and j that maximize F₁(i,j) or {circumflex over(F)}₁(i,j), may include one or more optimization processes (e.g.,constrained optimization processes, etc.), that determine the integervalues of indices i and j that maximize F₁(i,j) or {circumflex over(F)}₁(i,j) for the selected numerical input feature subject to one ormore constraints on a composition of the training samples associatedwith the selected numerical input feature, or on a magnitude of themaximized values of F₁(i,j) or {circumflex over (F)}₁(i,j).

For example, if the number B(v,s) of the sorted training samples havingnumerical feature values that exceed the corresponding numerical featurevalues having the i^(th) largest magnitude, and having Shapley featurevalues that exceed the threshold Shapley value s*=s^((j)), fails toinclude at least a threshold number B_(min) of the sorted trainingsamples for a particular combination of indices i and j, executednumerical-feature training module 232 may skip any computation ofF₁(i,j) for that particular combination of indices i and j. Similarly,if a number B(v,s) of the sorted training samples having numericalfeature values that exceed the corresponding numerical feature valueshaving the i^(th) smallest magnitude, and having Shapley feature valuesthat exceeds the threshold Shapley value s*=s^((j)), fails to include atleast the threshold number B_(min) of the sorted training samples for aparticular combination of indices i and j, executed numerical-featuretraining module 232 may skip any computation of {circumflex over(F)}₁(i,j) for that particular combination of indices i and j. Further,in some examples, numerical-feature training module 232 may performoperations that discard any computed value of F₁(i,j) or {circumflexover (F)}₁(i,j) that fails to exceed a predetermined threshold valueF_(min).

By way of example, and subject to these constraints, executednumerical-feature training module 232 may implement one or more of theoptimization processes to determine value of indices i and j thatmaximize the computed value of F₁(i,j) or alternatively, the computedvalue of {circumflex over (F)}₁(i,j), Based on the determination of theinteger values of indices i and j, executed numerical-feature trainingmodule 232 may establish the corresponding one of candidate Shapleythreshold values s^((j)) as the threshold Shapley value s* for theselected numerical input feature, and establish either the correspondingone of the numerical feature values having the i^(th) largest magnitude(e.g., when the determined indices i and j result in a maximum value ofF₁(i,j)), or the corresponding one of the numerical feature valueshaving the i^(th) smallest magnitude (e.g., when the determined indicesi and j result in a maximum value of {circumflex over (F)}₁(i,j)), asthe threshold feature value v* for the selected numerical feature.

Executed numerical-feature training module 232 may perform operationsthat package a feature identifier 234 of the selected numerical feature(e.g., an alphanumeric character string, etc.) and threshold data 236that specifies the threshold feature value v* and the threshold Shapleyvalue s* for the selected numerical input feature into correspondingportions of an element 238 of numerical feature parameter data 240.Further, executed numerical-feature training module 232 may also performoperations that generate one or more elements of predicted-positive data242 that characterizes an occurrence of a predicted positive for theselected numerical feature and that specifies textual content associatedwith the occurrence of the predicted positive.

By way of example, if the determined indices i and j were to result in amaximum value of F₁(i,j)), the predicted positive for the selectednumerical feature may occur when the numerical feature values exceed thethreshold feature value v* (e.g., v>v*). In some instances, executednumerical-feature training module 232 may package, intopredicted-positive data 242, an indicator of the predicted positivecondition (e.g., v>v*) and textual content that characterizes, orexplains, the predicted positive condition (e.g., “feature value beingtoo high”). Alternatively, if the determined indices i and j were toresult in a maximum value of {circumflex over (F)}₁(i,j), the predictedpositive for the selected numerical feature may occur when the numericalfeature values fails to exceed the threshold feature value v* (e.g.,v≤v*), and executed numerical-feature training module 232 may package,into predicted-positive data 242, an indicator of the predicted positivecondition (e.g., v≤v*) and textual content that characterizes, orexplains, the predicted positive condition (e.g., “feature value beingtoo low”). Executed numerical-feature training module 232 may alsoperform operations that incorporate predicted-positive data 242 into acorresponding portion of element 238.

Further, although not illustrated in FIG. 2B, executed numerical-featuretraining module 232 may also perform any of the exemplary processesdescribed herein to determine, for additional, or alternate, ones of thenumerical input values specified within process input data 175B andassociated with corresponding ones of the feature-specific subsets ofthe training samples, a corresponding threshold feature value v* andthreshold Shapley value s*, e.g., subject to the exemplary constraintsdescribed herein. Further, and for each of the additional, or alternate,ones of the numerical input values, executed numerical-feature trainingmodule 232 may generate an additional element of numerical featureparameter data that includes a corresponding feature identifier,threshold data includes the corresponding threshold feature value v* andthreshold Shapley value s*, and corresponding elements ofpredicted-positive data.

As described herein, the elements of process input data 175B mays alsospecify one or more categorical features, and a categorical-featuretraining module 244 of executed training engine 230 parse the featureidentifiers maintained within the elements of sampling data 220 toobtain a plurality of the additional training samples associated witheach, or a targeted subset of, the categorical input features specifiedby the elements of process input data 175B. In some instances, executedcategorical-feature training module 244 may perform any of the examplesdescribed herein to sort training samples into feature-specific subsetsassociated with corresponding ones of the categorical input features,and further, that sort the training samples within each of thefeature-specific subsets in accordance with the Shapley feature values(e.g., in descending order based on the corresponding Shapley featurevalues, etc.). In some instances, executed categorical-feature trainingmodule 244 may select one of the categorical input features for training(e.g., categorical feature associated with a corresponding featureidentifier f_(ID)) and may obtain the sorted training samples maintainedwithin the feature-specific subset associated with the selectedcategorical input feature (e.g., a plurality of N training samples{(f_(ID),v_(i),s_(i))}_(i=1) ^(N), where v_(i) corresponds to thefeature value of the selected categorical input feature within thei^(th) training sample, and s_(i) corresponds to the Shapley featurevalue of the selected categorical input feature within the i^(th)training sample).

As described herein, the categorical feature values v_(i) maintainedwith the sorted training samples may specify one of a plurality ofcandidate categories associated with the selected categorical inputfeature (including, in some instances, a null value indicating anabsence of a category, e.g., due to a missing one of the categoricalfeature values v_(i) in one or more of the sorted training samples). Byway of example, and for the selected categorical input feature, executedcategorical-feature training module 244 may parse the categoricalfeature values v_(i) maintained within the sorted training samples toidentify each of the candidate categories associated with the selectedcategorical input feature (including the null value described herein),although in other instances (not illustrated in FIG. 2B), executedcategorical-feature training module 244 may access elements of data thatidentify, and characterize, each of the candidate categories associatedwith the selected categorical input feature.

In some instances, in training further the Shapley splitter process,executed categorical-feature training module 244 may perform operationsthat determine the threshold category c* and the threshold Shapley values* for the selected categorical input feature that maximize the valuesof precision and recall for the selected categorical input feature andas such, that maximize the F₁ score associated with the selectedcategorical feature. By way of example, and through an implementation ofone or more of the exemplary training processes described herein,executed categorical-feature training module 244 the threshold categoryc* and the threshold Shapley value s* for the selected categorical inputfeature in accordance with:

${\left( {c^{*},s^{*}} \right) = {\underset{{({c,s})} \in R^{2}}{\arg\max}\frac{2{B\left( {c,s} \right)}}{{2{B\left( {c,s} \right)}} + {A\left( {c,s} \right)} + {D\left( {c,s} \right)}}}},$

where: A(c,s) corresponds to a number of the sorted training sampleshaving a categorical feature value v_(i) inconsistent with the thresholdcategory c* and a Shapley feature value s_(i) that exceeds the thresholdShapley value s*; B(v,s) corresponds to a number of the sorted trainingsamples having a categorical feature value v_(i) that is consistent withthe threshold category c* and a Shapley feature value s_(i) that exceedsthe threshold Shapley value s*; and D(v,s) corresponds to a number ofthe sorted training samples having a categorical feature value v_(i)that is consistent with the threshold category c* and a Shapley featurevalue s_(i) that fails exceeds the threshold Shapley value s*. Forexample, a categorical feature value may be consistent with thethreshold category c* when that categorical feature value includes, andcorresponds to, the threshold category c*, and a categorical featurevalue may be inconsistent with the threshold category c* when thatcategorical feature value fails to include, and fails to correspond to,the threshold category c*.

Further, and for each of the candidate categories c associated with theselected categorical input feature (including, in some instances, thenull value described herein), executed categorical-feature trainingmodule 244 may also: (i) establish F₁(i,c) as the F₁ score computedbased on one or more of the sorted training samples having categoricalfeature values that include the corresponding candidate category c; and(ii) establish {circumflex over (F)}₁(i,c) as the F₁ score computedbased on one or more of the sorted training samples having categoricalfeature values fail to include the corresponding candidate category c.In some instances, executed categorical-feature training module 244 mayalso perform operations to determine the integer value of index i (e.g.,ranging from unity to N) and a corresponding one of candidate categoriesc that result in a maximum value of F₁(i,c) or alternatively, a maximumvalue of {circumflex over (F)}₁(i,c), and through an implementation ofone or more of these exemplary training processes, categorical-featuretraining module 244 may compute the values of F₁(i,c) and {circumflexover (F)}₁(i,c) under an assumption that a top number k of the Shapleyfeature values of the sorted training samples predict positive (e.g.,using s*=s_(k), where k∈[1,N]).

In some instances, the operations performed by executedcategorical-feature training module 244, which determine the integervalue of index i and j that maximize F₁(i,j) or {circumflex over(F)}₁(i,j), may include one or more optimization processes (e.g.,constrained optimization processes, etc.) that determine the integervalues of indices i and the corresponding one of candidate categories cthat maximize F₁(i,j) or {circumflex over (F)}₁(i,j) for the selectedcategorical input feature subject to one or more constraints on index i,a composition of the training samples associated with the selectedcategorical input feature, or on a magnitude of the maximized values ofF₁(i,j) or {circumflex over (F)}₁(i,j). For example, when calculatingthe values of F₁(i,c) and {circumflex over (F)}₁(i,c) for correspondingones of the candidate categories c, executed categorical-featuretraining module 244 may iterate across values of index i that fail toexceed that the value of s_(m), as described herein.

Further, if the number B(v,s) of the sorted training samples havingcategorical feature values that include a corresponding candidatecategory c, and having Shapley feature values that exceed the thresholdShapley value s*=s^((j)), fails to include at least a threshold numberB_(min) of the sorted training samples for a particular combination ofindex i and candidate category c, executed categorical-feature trainingmodule 244 may skip any computation of F₁(i,j) for that particularcombination of index i and candidate category c. Similarly, if a numberB(v,s) of the sorted training samples having categorical feature valuesthat fail to include a corresponding candidate category c, and havingShapley feature values that exceeds the threshold Shapley value s*=s(i),fails to include at least the threshold number B_(min) of the sortedtraining samples for a particular combination of index i and candidatecategory c, executed categorical-feature training module 244 may skipany computation of {circumflex over (F)}₁(i,j) for that particularcombination of index i and candidate category c. In some instances,categorical-feature training module 244 may also perform operations thatdiscard any computed value of F₁(i,j) or {circumflex over (F)}₁(i,j)that fails to exceed a predetermined threshold value F_(min).

By way of example, and subject to these constraints, executedcategorical-feature training module 244 may implements on or more of theoptimization processes to determine value of index i and candidatecategory c that maximize the computed value of F₁(i,j) or alternatively,the computed value of {circumflex over (F)}₁(i,j), Based on thedetermination of the integer values of index i and candidate category c,executed categorical-feature training module 244 may establish Shapleyfeature value s_(i) the threshold Shapley value s* for the selectedcategorical input feature, and establish candidate category c as thethreshold category c*for the selected categorical feature. Executedcategorical-feature training module 244 may also perform operations thatpackage a feature identifier 246 of the selected categorical feature(e.g., an alphanumeric character string, etc.) and threshold data 248that specifies the threshold category c* and the threshold Shapley values* for the selected categorical input feature into correspondingportions of an element 250 of categorical feature parameter data 252.

Further, executed categorical-feature training module 244 may alsoperform operations that generate one or more elements ofpredictive-positive data 254 that characterize an occurrence of apredicted positive based on the application of the trained,gradient-boosted, decision-tree process to the categorical featurevalues of the selected numerical feature, and to packagepredictive-positive data 254 into a corresponding portion of element250. By way of example, if the determined index i and candidate categoryc were to result in a maximum value of F₁(i,c)), the predicted positivefor the selected categorical feature may occur when the categoricalfeature values are consistent with the threshold category c* (e.g.,v=c*). Alternatively, if the determined index i and candidate category cwere to result in a maximum value of {circumflex over (F)}₁(i,c), thepredicted positive for the selected numerical feature may occur when thecategorical feature values are inconsistent with the threshold categoryc* (e.g., v≠c*).

Further, although not illustrated in FIG. 2B, executedcategorical-feature training module 244 may also perform any of theexemplary processes described herein to determine, for additional, oralternate, ones of the categorical input values specified within processinput data 175B and associated with corresponding ones of thefeature-specific subsets of the training samples, a correspondingthreshold category c* and threshold Shapley value s*, e.g., subject tothe exemplary constraints described herein. Further, and for each of theadditional, or alternate, ones of the categorical input features,executed categorical-feature training module 244 may generate anadditional element of categorical feature parameter data 252 thatincludes a corresponding feature identifier, threshold data includes thecorresponding threshold category c* and threshold Shapley value s*, andfurther data that characterizes an occurrence of a predicted positivebased on the application of the trained, gradient-boosted, decision-treeprocess to the numerical feature values of the additional, or alternate,one of selected categorical feature.

C. Exemplary Techniques for Applying Trained Explainability Processes toPredicted, Customer-Specific Output of Trained, Gradient-Boosted,Decision-Tree Processes

In some instances, described herein, a machine-learning orartificial-intelligence process, such as a gradient-boosteddecision-tree process, may be trained to predict, at a temporalprediction point, a likelihood of an occurrence of one or more eventsassociated with, or involving, a customer of the financial institutionduring a future temporal interval using training data associated with afirst prior temporal interval, and using validation data associated witha second, and distinct, prior temporal interval. Further, and based onan application of the trained gradient-boosted, decision-tree process toinput datasets associated with one or more customers of the financialinstitution, the one or more distributed components of FI computingsystem 130 may generate elements of output data indicative of alikelihood of an occurrence of one or more events involvingcorresponding ones of the customers and the corresponding financialproduct or service during a future temporal interval disposed subsequentto a prediction date. The generated elements of output data may include,for corresponding ones of the customers, a numerical value indicative ofa predicted likelihood of the future occurrence of the one of moreevents, and in some instances, the elements of customer-specific outputdata may inform an implementation by the financial institution of one ormore risk management, risk mitigation, or collections strategiesinvolving corresponding ones of the customers.

For example, the one or more distributed components of FI computingsystem 130 may perform any of the exemplary processes described hereinto generate input datasets associated with all, or a selected subset, ofthe customers of the financial institution, and to apply the trained,gradient-boosted, decision-tree process described herein, to each of theinput datasets. The selected subset may include one or more customers ofthe financial institution that hold a credit product issued by thefinancial institution, such as, but not limited to, the secured orunsecured credit-card accounts described herein, and in some instances,the one or more distributed components of FI computing system 130 mayperform any of the exemplary processes described herein to generateinput datasets associated with the selected subset of the customers ofthe financial institution, and to apply the trained machine-learning orartificial-intelligence process to each of the input datasets inaccordance with a predetermined temporal schedule (e.g., on a daily,weekly, or monthly basis), or in response to a detection of a triggeringevent (e.g., based on the usage of the credit-card account or based on arequest by a customer to modify a term or condition of the credit-cardaccount). As described herein, each of the generated elements of outputdata may include a numerical score (e.g., either zero or unity)indicative of a predicted likelihood that a corresponding one of thecustomers will be involved in a default event during the future temporalinterval, e.g., with a score of zero being indicative of a predictednon-occurrence of the future default event, and with a score of unitybeing indicative of a predicted occurrence of the future default event.

In some instances, the generated elements of output data, e.g., thenumerical scores, may classify the customers of the financialinstitution based on the predicted likelihood of their involvement inthe future occurrences of the default events, and the elements ofcustomer-specific output data may inform not only a determination by thefinancial institution of an initial set of terms and conditionsassociated with a newly issued financial product (e.g., a credit-cardaccount, etc.), but may also inform decisions, by the financialinstitution, to approve or decline requests for modifications to aninitial set of terms and conditions, or to authorize a transactioninvolving the issued financial product, as well as decisions, by thefinancial institution, to suspend, close, or subsequently reissue thecredit product, and decisions to implement one or more collectionprocesses or strategies involving the financial product. For instance,and as described herein, FI computing system 130 may perform operationsthat, in conjunction with one or more computing systems of the financialinstitution, modify a term or condition of a product or service (e.g., acredit-card account, etc.) held by one or more of the selected subset ofthe customers based on the predicted likelihood of the involvement ofthese customers in the future occurrences of the default events.

For example, a customer of the financial institution may request anincrease in a credit limit associated with a credit-card account issuedby the financial institution. A device operable by, or associated with,the customer may execute one or more application programs (e.g., a webbrowser or mobile application associated with the financialinstitution), and the executed application program may generate elementsof data that identify and characterize the customer and the requestedcredit-card account, and may perform operations that cause the device totransmit the generated elements of data across a communications network,such as network 120, to one or more additional computing systems of thefinancial institution, such as an issuer system associated with thecredit-card account.

In some instances, and prior to implementing the requested increased tothe credit limit, the issuer system may provision data identifying thecustomer to FI computing system 130, e.g., across network 120. The oneor more distributed components of FI computing system 130 may performany of the exemplary processes described herein to generate an inputdataset associated with the customer (e.g., in accordance with theelements of process input data 175B), to apply the trainedgradient-boosted, decision-tree process to the generated input dataset(e.g., in accordance with the elements of process parameter data 175A),and based on the application of the trained gradient-boosted,decision-tree process to the input dataset, generate an element ofoutput data (e.g., the numerical score described herein) that indicatesa predicted likelihood of an occurrence of a default event involving thecustomer during the future temporal interval. Further, and concurrentlywith the application of the trained gradient-boosted, decision-treeprocess to the input dataset, the one or more distributed components ofFI computing system 130 may also perform any of the exemplary processesdescribed herein to apply to the input dataset one or moreexplainability processes, such as, but not limited to, the trainedShapley splitter process described herein.

Based on the application of the explainability processes to the inputdataset, the one or more distributed components of FI computing system130 may perform operations, described herein, that generate elements ofelements of natural language that characterize a causal relationshipbetween the value of one or more input features within acustomer-specific input dataset on a magnitude of a corresponding,customer-specific element of predicted output data, and that provisionthe customer-specific element of predicted output, and the correspondingelements of natural language, to the issuer system. Certain of theexemplary processes described herein provide, in real-time andcontemporaneously with the requested credit-limit increase, anindication to the issuer system of the likelihood of the future defaultevent involving the customer and the credit-card account, and based onthe provisioned element of output data, the issuer system may elect toapprove the requested credit-limit increase (e.g., to issue a “positive”decision) or alternatively, to decline the requested credit-limitincrease (e.g., to issue an “adverse” decision). Further, the elementsof natural language may characterize one or more reasons for the adversedecision (or alternatively, the positive decision) regarding therequested credit-limit increase, and when provisioned to the customerdevice for presentation in a digital interval, may enable to customer toappreciate readily the reasons for the adverse (or positive) decision.

Referring to FIG. 3A, aggregated data store 132 of FI computing system130 may maintain one or more elements of customer data 302. In someinstances, each of the one or more elements of customer data 302 may beassociated with a customer of the financial institution that holds one,or more issued financial products, such as one or more secured orunsecured credit-card accounts. The disclosed embodiments are, however,not limited to the exemplary credit-card accounts described herein, andin other instances, the elements of customer data 302 may be associatedwith customers of the financial institution that hold additional, oralternate, financial products issued by the financial institution, suchas, but not limited to, a secured financial product (e.g., a homemortgage, an automobile loan, etc.) or another, unsecured credit product(e.g., an unsecured personal loan, an unsecured line-of-credit, etc.).

FI computing system 130 may, for example, receive all, or a selectedportion, of customer data elements 302 from one or more issuer systemsassociated with the credit-card accounts, such as, but not limited to,issuer system 301 of FIG. 3A. In some instances, issuer system 301 mayrepresent a computing system that includes one or more servers andtangible, non-transitory memories storing executable code andapplication modules. Further, the one or more servers may each includeone or more processors (such as a central processing unit (CPU)), whichmay be configured to execute portions of the stored code or applicationmodules to perform operations consistent with the disclosed embodiments.Issuer system 301 may also include a communications interface, such asone or more wireless transceivers, coupled to the one or more processorsfor accommodating wired or wireless internet communication with othercomputing systems and devices operating within environment 100. In someinstances, issuer system 301 may be incorporated into a respective,discrete computing system, although in other instances, issuer system301 may correspond to a distributed computing system having a pluralityof interconnected, computing components distributed across anappropriate computing network, such as communications network 120 ofFIG. 1A, or to a publicly accessible, distributed or cloud-basedcomputing cluster.

Referring back to FIG. 3A, an application program executed by the one ormore processors of issuer system 301, may transmit portions of customerdata elements 302 across network 120 to FI computing system 130. Thetransmitted portions may be encrypted using a corresponding encryptionkey, such as a public cryptographic key associated with FI computingsystem 130, and a programmatic interface established and maintained byFI computing system 130, such as application programming interface (API)304, may receive the portions of customer data 302 from issuer system301, or from additional, or alternate, ones of the issuer systems.

API 304 may, for example, route each of the elements of customer data302 to executed data ingestion engine 136, which may perform operationsthat store the elements of customer data 302 within one or moretangible, non-transitory memories of FI computing system 130, such aswithin aggregated data store 132. In some instances, and as describedherein, the received elements of customer data 302 may be encrypted, andexecuted data ingestion engine 136 may perform operations that decrypteach of the encrypted elements of customer data 302 using acorresponding decryption key (e.g., a private cryptographic keyassociated with FI computing system 130) prior to storage withinaggregated data store 132. Further, although not illustrated in FIG. 3A,aggregated data store 132 may also store one or more additional elementsof customer data identifying customers of the financial institution thathold corresponding ones of the unsecured credit products, and executeddata ingestion engine 136 may perform one or more synchronizationoperation that merge the received elements of customer data 302 with thepreviously stored elements of customer data, and that eliminate anyduplicate elements existing among the received elements of customer data302 with the previously stored elements of customer data (e.g., throughan invocation of an appropriate Java-based SQL “merge” command).

As described herein, each of the elements of customer data 302 may beassociated with, and include a unique identifier of a customer of thefinancial institution, and FI computing system 130 may receive each ofthe elements of customer data 302 from a corresponding one of issuersystems 301, such as issuer system 301. For example, as illustrated inFIG. 3A, element 306 of customer data 302, which may be associated witha particular one of the customers and received from issuer system 301,may include a customer identifier 308 assigned to the particularcustomer by FI computing system 130 (e.g., an alphanumeric characterstring, etc.), and a system identifier associated with issuer system 301(e.g., an Internet Protocol (IP) address, a media access control (MAC)address, etc.). Further, although not illustrated in FIG. 3A, eachadditional, or alternate, element of customer data 302 may be associatedwith an additional customer of the financial institution that holds anunsecured credit product and received from a corresponding one of issuersystems 301, and may include a customer identifier associated with thatadditional customer and a system identifier associated with thecorresponding one of issuer systems 301.

As described herein, FI computing system 130 may perform any of theexemplary processes described herein to generate an input datasetassociated with each of the customers identified by the discreteelements of customer data 302, and to apply the trained,gradient-boosted, decision-tree process described herein to each of theinput datasets, in accordance with a predetermined temporal schedule(e.g., on a daily, weekly, or monthly basis, etc.), or in response to adetection of a triggering event. By way of example, the triggering eventmay correspond to a detected change in a composition of the elements ofcustomer data 302 maintained within aggregated data store (e.g., to aningestion of additional elements of customer data 302, etc.) or to areceipt of an explicit request received from one or more of issuersystems 301.

In some instances, and in accordance with the predetermined temporalschedule, or upon the detection of the triggering event, a process inputengine 312 executed by FI computing system 130 may perform operationsthat access the elements of customer data 302 maintained withinaggregated data store 132, and that obtain the customer identifiermaintained within a corresponding one of the accessed elements ofcustomer data 302. For example, as illustrated in FIG. 3A, executedprocess input engine 312 may access element 306 of customer data 302(e.g., as maintained within aggregated data store 132) and obtaincustomer identifier 308, which includes, but is not limited to, thealphanumeric character string assigned to the particular customer of thefinancial institution (e.g., one of customer identifiers 146 and 156 ofFIGS. 1A, and 1B, as described herein).

Executed process input engine 312 may also access consolidated datastore 144, and perform operations that identify, within consolidateddata records 314, a subset 316 of consolidated data records that includecustomer identifier 308 and as such, are associated with the particularcustomer of the financial institution identified by element 306 ofcustomer data 302. As described herein, each of consolidated datarecords 314 may be associated with a customer of the financialinstitution, and may characterize that customer, the interaction of thatcustomer with the financial institution and with other financialinstitutions, and any associated default events involving that customerduring a corresponding temporal interval. For example, and as describedherein, each of consolidated data records 314 may include acorresponding customer identifier (e.g., an alphanumeric characterstring assigned to a corresponding customer), a corresponding temporalidentifier (e.g., that identifies the corresponding temporal interval),and one or more consolidated data elements associated with thecorresponding customer. Examples of these consolidated data elements mayinclude, but are not limited to, elements customer profile data, accountdata, delinquency data, or credit-bureau data, which may be ingested,processed, aggregated, or filtered by FI computing system 130 using anyof the exemplary processes described herein.

In some instances, and as illustrated in FIG. 3A, each data recordwithin subset 316 may include customer identifier 308 and as such, maybe associated with the particular customer identified by element 306 ofcustomer data 302. Each of subset 316 of consolidated data records 314may also include a temporal identifier of a corresponding temporalinterval, and one or more consolidated elements associated with theparticular customer, the interaction of particular customer with thefinancial institution and with other financial institutions, and anyassociated default events involving the particular customer duringcorresponding ones of the temporal intervals. By way of example, datarecord 318 of subset 316 may include customer identifier 308, acorresponding temporal identifier 320 (e.g., “2021-11-30,” indicating atemporal interval spanning Nov. 1, 2021, through Nov. 30, 2021), andconsolidated data elements 322, which identify and characterize theparticular customer during the temporal interval.

Executed process input engine 312 may also perform operations thatobtain, from consolidated data store 144, elements of process input data175B that characterizes a composition of an input dataset for thetrained, gradient-boosted, decision-tree process. In some instances,executed process input engine 312 may parse process input data 175B toobtain the composition of the input dataset, which not only identifiesthe elements of customer-specific data included within each input dataset dataset (e.g., feature identifiers of numerical or categorical inputfeature values, as described herein), but also a specified sequence orposition of these input feature values within the input dataset. Basedon the parsed portions of process input data 175B, executed processinput engine 312 may perform operations that identify, and obtain orextract, one or more of the input feature values from one or more ofdata records maintained within subset 316 of consolidated data records314 and associated with temporal intervals disposed within theextraction interval Δt_(extract), as described herein, and further thatcompute one or more of the input features values based on the elementsof extracted or obtained data. Executed process input engine 312 mayperform operations that package the obtained, or extracted, inputfeature values within a corresponding one of input datasets 328, such asinput dataset 330 associated with the particular customer identified byelement 306 of customer data 302, in accordance with their respective,specified sequences or positions.

Through an implementation of these exemplary processes, executed processinput engine 312 may populate an input dataset associated with theparticular customer identified by element 306 of customer data 302, suchas input dataset 330 of input datasets 328, with input feature valuesobtained or extracted from, or computed, determined or derived from,elements of data within the data records of subset 316. Further, in someinstances, executed process input engine 312 may also perform any of theexemplary processes described herein to generate, and populate withinput feature values, an additional one of input datasets 328 for eachof the additional, or alternate, customers of the financial institution(e.g., which are associated with additional, or alternate, elements ofcustomer data 302). Executed process input engine 312 may package eachof the customer-specific input datasets within input datasets 328, andexecuted process input engine 312 may provide input datasets 328 as aninput to a predictive engine executed by the one or more processors ofFI computing system 130, such as executed predictive engine 214.

As illustrated in FIG. 3A, executed predictive engine 214 may performoperations that obtain, from consolidated data store 144, processparameter data 175A that includes one or more process parameters of thetrained, gradient-boosted, decision-tree process, such as one or more ofthe exemplary process parameters described herein. In some instances,and based on portions of process parameter data 175A, executedpredictive engine 214 may perform operations that establish a pluralityof nodes and a plurality of decision trees for the trained,gradient-boosted, decision-tree process, each of which receive, asinputs (e.g., “ingest”), corresponding elements of input datasets 328.Further, and based on the execution of predictive engine 214, and on theingestion of input datasets 328 by the established nodes and decisiontrees of the trained, gradient-boosted, decision-tree process, FIcomputing system 130 may perform operations that apply the trained,gradient-boosted, decision-tree process to each of the input datasets ofinput datasets 328, including input dataset 330, and that generate anelement of output data 334 associated with a corresponding one of inputdatasets 328, and as such, a corresponding one of the customersidentified by the elements of customer data 302. For example, outputdata 334 may include an element 336 associated with input dataset 330and with the customer identified by element 306 of customer data 302.

By way of example, and as described herein, each of the generatedelements of output data 334 may include a numerical score indicative ofa predicted likelihood that the corresponding one of the customers willbe involved in a default event during the future temporal interval(e.g., the target interval Δt_(target), described herein). In someinstances, the numerical score within each of the elements of outputdata 334 may correspond to either zero or unity, with a score of zerobeing indicative of a predicted non-occurrence of the default eventduring the future temporal interval, and with a score of unity beingindicative of a predicted occurrence of the default event during thefuture temporal interval. Executed predictive engine 214 may provide thegenerated elements of output data 334 (e.g., either alone, or inconjunction with corresponding ones of input datasets 328) as an inputto a post-processing engine 338 executed by the one or more processorsof FI computing system 130.

The one or more processors of FI computing system 130 may also performoperations that, either concurrently with, or subsequent to, theapplication of the trained, gradient-boosted, decision-tree process toinput datasets 328 by executed predictive engine 214, generate, foreach, or a selected subset, of input datasets 328, a plurality ofdiscrete elements of textual content that characterize an impact of oneor more numerical or categorical feature values on a correspondingelement of output data 334. By way of example, and for a correspondingone of input datasets 328, such as input dataset 330, the one or moreprocessors of FI computing system 130 may perform any of the exemplaryprocesses described herein to compute, for each of values of the inputfeatures (e.g., as specified within process input data 175B), a metricvalue that characterizes a contribution of the input feature value tothe predicted output of the trained, gradient-boosted decision-treeprocess, such as, but not limited to, a Shapley feature value. Further,in some examples, and based on the computed Shapley feature values, theone or more processors of FI computing system 130 may performoperations, described herein, to select a subset of the input featurevalues of input dataset 330 (e.g., a predetermined number of the inputfeature values associated with the largest Shapley feature values, etc.)and generate, for each of the subset of the input feature values,elements of textual content that identify and characterize afeature-specific reason for the corresponding element of predictedoutput data 334.

As described herein, the elements of textual content associated with aparticular one of the input feature values of input dataset 330, and ofother ones of input datasets 328, may specify, among other things, thatthe particular input feature value is “too low” or “too high” (e.g., afeature-specific reason associated with a numerical input feature) orthat the particular input feature value is, or is not, associated with athreshold category (e.g., a feature-specific reason associated with acategorical input feature). In some instances, the one or moreprocessors of FI computing system 130 may perform operations that mapthe elements of textual content, and the corresponding feature-specificreasons, to corresponding elements of natural that characterize thefeature-specific reason (e.g., to generate adverse reasons), and to thefinancial institution and its customers, and that provision dataspecifying at least a subset of the adverse reasons, and a correspondingelement of output data 334, to issuer system 203.

Referring to FIG. 3B, an explainability engine executed by the one ormore processors of FI computing system 130, such as executedexplainability engine 210, may obtain one or more of input datasets 328,such as input dataset 330, and may obtain an element of output data 334associated with each of the one or more of input datasets 328, such asoutput data element 336. In some instances, and based on input datasets328 and output data 334, executed explainability engine 210 may performany of the exemplary processes described herein, in conjunction withexecuted predictive engine 214, to compute Shapley feature values thatcharacterize a contribution of corresponding ones of the input features(e.g., the numerical and categorical input features described herein) tothe outcome of the trained, gradient-boosted, decision-tree process(e.g., the numerical values indicative predicted likelihood of anoccurrence of a default involving a corresponding customer and acredit-card account during a future temporal interval, as maintainedwithin the elements of output data 334). Executed explainability engine210 may, for example, associated each of the computed Shapley featurevalues with an identifier of the corresponding numerical or categoricalinput feature (e.g., one of the exemplary feature identifiers maintainedwithin process input data 175B), and may rank each of the associatedpairs feature identifiers and Shapley feature values based oncorresponding ones of Shapley feature values (e.g., in descendingorder), and package ranked pairs 340 of feature identifiers (e.g.,f_(ID,i)) and Shapley feature values (e.g., s_(i)) into correspondingportions of explainability data. (f_(ID,i),s_(i))

Further, executed explainability engine 210 may provision theexplainability data 342, including the ranked pairs of featureidentifiers and Shapley feature values, to a reason generation engine344 executed by the one or more processors of FI computing system 130(e.g., based on a programmatic signal generated by executedexplainability engine 210). In some instances, and based onexplainability data 342, executed reason generation engine 344 mayperform any of the exemplary processes described herein to generateelements of textual content that identify and characterize afeature-specific reason associated with each, or a selected subset, ofthe input feature values maintained within corresponding ones of inputdatasets 328 and as such, with corresponding element of predicted outputdata 334. For example, as illustrated in FIG. 3B, a selection module 346of executed reason generation engine 344 may receive explainability data342, and may perform operations that parse ranked pairs 340 of featureidentifiers and Shapley feature values, and extract a subset 348 ofranked pairs 340 associated with those numerical or categorical inputfeatures characterized by the largest of the ranked Shapley featurevalues. For example, extracted subset 348 may include a predeterminednumber of ranked pairs 340 of feature identifiers and Shapley featurevalues, and executed selection module 346 may provide subset 348, andinput datasets 328, as an input to Shapley-splitter predictive module350 of executed reason generation engine 344, which may performoperations that apply the trained, Shapley-splitter process describedherein to one or more of the input feature values maintained withincorresponding ones of input datasets 328, and generate, elements oftextual content that identify and characterize a feature-specific reasonfor the corresponding element of predicted output data 334.

By way of example, as illustrated in FIG. 3B, executed Shapley-splitterpredictive module 350 may perform operations that access a ranked pairof subset 348, such as pair 348A that includes a feature identifierf_(ID,1) of a particular numerical input feature and an associatedShapley feature value s₁, and obtain an element 352 of numerical featureparameter data 240 that includes the feature identifier f_(ID,1) assuch, is associated with the particular numerical input feature.Executed Shapley-splitter predictive module 350 may obtain, from element352, threshold data 354 includes a threshold feature value v* and athreshold Shapley value s* for the particular numerical input featureand elements of predicted-positive data 356 for the particular numericalinput feature. Further, and based on feature identifier f_(ID,1) and theelements of process input data 175B, executed Shapley-splitterpredictive module 350 may also determine a position of the feature valueof the particular numerical input feature within input dataset 330, andmay perform operations that obtain the feature value of the particularnumerical input feature (e.g., feature value v₁) from the determinedposition within input dataset 330.

In some instances, executed Shapley-splitter predictive module 350 mayalso perform operations that determine whether the Shapley feature values₁ exceeds the threshold Shapley value s* for the particular numericalinput feature (e.g., that s₁>s₁*), and further, whether the featurevalue v₁ of the particular numerical input feature, as maintained withininput dataset 330, satisfies the predicted-position condition for theparticular numerical input feature value, as specified withinpredicted-positive data 356. For example, and based on portions ofpredicted-positive data 356, Shapley-splitter predictive module 350 mayestablish that a predicted positive for the particular numerical inputfeature occur when a corresponding feature value exceeds the thresholdfeature value v* (e.g., v>v*), and that the feature value v₁ satisfiesthe predicted-positive condition for the particular numerical inputfeature when the feature value v₁ exceeds the threshold feature valuev₁.

By way of example, as illustrated in FIG. 3B, executed Shapley-splitterpredictive module 350 may determine that (i) the Shapley feature values₁ exceeds the threshold Shapley value s* for the particular numericalinput feature and (ii) feature value v₁ exceeds the threshold featurevalue v* for the particular numerical input feature. Based on thedetermination, executed Shapley-splitter predictive module 350 mayperform operations that apply the trained Shapley-splitter process tothe feature value v₁ of the particular numerical input feature. Forinstance, and based on the determination that the feature value v₁ ofthe particular numerical input exceeds the threshold feature value v₁*,executed Shapley-splitter predictive module 350 may processpredicted-positive data 356 and establish, for the satisfiedpredicted-positive condition (e.g., v₁>v₁*), that a feature-specificreason characterizing, or explaining, the predicted-positive conditionincludes the phrase “the feature value is too high.” ExecutedShapley-splitter predictive module 350 may perform operations thatpackage the phrase “the feature value is too high” into and element 358Aof textual content 358 (e.g., as the corresponding feature-specificreason).

In other examples, not illustrated in FIG. 3B, predicted-positive data356 may specify that the predicted positive for the particular numericalinput feature occurs when a corresponding feature value fails to exceedthe threshold feature value v* (e.g., v≤v*). Based on a determinationthat the Shapley feature value s₁ exceeds the threshold Shapley value s*for the particular numerical input feature, and that the feature valuev₁ satisfies the additional predicted-positive condition for theparticular numerical input feature (e.g., that the feature value v₁fails to exceeds the threshold feature value v₁*), executedShapley-splitter predictive module 350 may process predicted-positivedata 356 and establish, for the additional predicted-positive condition(e.g., v₁≤v₁*), that a feature-specific reason characterizing, orexplaining, the additional predicted-positive condition includes thephrase “the feature value is too low.” Executed Shapley-splitterpredictive module 350 may perform operations that package the phrase“the feature value is too low” into an additional, or alternate, portionof textual content associated with the particular numerical inputfeature (e.g., as the corresponding feature-specific reason).

Further, and as illustrated in FIG. 3B, executed Shapley-splitterpredictive module 350 may perform operations that access an additionalranked pair of subset 348, such as pair 348B that includes a featureidentifier f_(ID,2) of a particular categorical input feature and anassociated Shapley feature value s₂. In some instances, executedShapley-splitter predictive module 350 may parse the elements ofcategorical feature parameter data 252, determine that element 360includes feature identifier f_(ID,2) of particular the categorical inputfeature, and obtain, from element 360, threshold data 362 includes athreshold category c* and a threshold Shapley value s* for theparticular numerical input feature and elements of predicted-positivedata 364 for the particular categorical input feature. Further, andbased on feature identifier f_(ID,2) and the elements of process inputdata 175B, executed Shapley-splitter predictive module 350 may alsodetermine a position of the feature value of the particular categoricalinput feature within input dataset 330, and may perform operations thatobtain the feature value of the particular categorical input feature(e.g., feature value v₂) from the determined position within inputdataset 330.

As described herein, executed Shapley-splitter predictive module 350 mayalso perform operations that determine whether the Shapley feature values₂ exceeds the threshold Shapley value s* for the particular categoricalinput feature (e.g., that s₂>s₂*), and further, whether the featurevalue v₂ of the particular numerical input feature, as maintained withininput dataset 330, satisfies the predicted-position condition for theparticular numerical input feature value, as specified withinpredicted-positive data 356. For example, and based on portions ofpredicted-positive data 364, executed Shapley-splitter predictive module350 may establish that a predicted positive for the particularcategorical input feature occurs when a corresponding feature value isconsistent with, or includes, the threshold category c* (e.g., c=c*),and that the feature value v₁ satisfies the predicted-positive conditionfor the particular categorical input feature when the feature value v₁is consistent with, or includes, the threshold category c*.

By way of example, as illustrated in FIG. 3B, executed Shapley-splitterpredictive module 350 may determine that (i) the Shapley feature values₂ exceeds the threshold Shapley value s* for the particular categoricalinput feature and (ii) feature value v₁ includes threshold category c*for the particular categorical input feature. Based on thedetermination, executed Shapley-splitter predictive module 350 mayperform operations that apply the trained Shapley-splitter process tothe feature value v₂ of the particular categorical input feature. Forinstance, executed Shapley-splitter predictive module 350 may processpredicted-positive data 364 and establish, for the satisfiedpredicted-positive condition (e.g., c₁=c*), that a feature-specificreason characterizing, or explaining, the predicted-positive conditionincludes the phrase “the feature value is the threshold category.”Executed Shapley-splitter predictive module 350 may perform operationsthat package the phrase “the feature value is the threshold category”into an element 358B of textual content 358 (e.g., as the correspondingfeature-specific reason).

In other examples, not illustrated in FIG. 3B, predicted-positive data364 may specify that the predicted positive for the particularcategorical input feature occurs when a corresponding feature valuefails to be consistent with, or include, the threshold category c*(e.g., v₂≠c*). Based on a determination that the Shapley feature values₁ exceeds the threshold Shapley value s* for the particular categoricalinput feature, and that the feature value v₂ satisfies the additionalpredicted-positive condition for the particular categorical inputfeature (e.g., that the feature value v₂ fails to include the thresholdcategory c*), executed Shapley-splitter predictive module 350 mayprocess predicted-positive data 364 and establish, for the additionalpredicted-positive condition (e.g., v₂ ≠c*), that a feature-specificreason characterizing, or explaining, the additional predicted-positivecondition includes the phrase “the feature value is not the thresholdcategory.” Executed Shapley-splitter predictive module 350 may performoperations that package the phrase “the feature value is not thethreshold category” into an additional, or alternate, portion of textualcontent associated with the particular numerical input feature (e.g., asthe corresponding feature-specific reason).

Further, executed Shapley-splitter predictive module 350 may performoperations that access an additional ranked pair of subset 348, such aspair 348C that includes a feature identifier f_(ID,3) of an additionalnumerical input feature and an associated Shapley feature value s₃, andobtain an additional element of numerical feature parameter data 240(not illustrated in FIG. 3B) that includes the feature identifierf_(ID,1) as such, is associated with the additional numerical inputfeature. As described herein, executed Shapley-splitter predictivemodule 350 may obtain, from the additional element, threshold dataincludes a threshold feature value v₃* and a threshold Shapley value s₃*for the additional numerical input feature and elements ofpredicted-positive data 356 for the particular numerical input feature.Further, and based on feature identifier f_(ID,3) and the elements ofprocess input data 175B, executed Shapley-splitter predictive module 350may also determine a position of the feature value of the particularnumerical input feature within input dataset 330, and may performoperations that obtain the feature value of the particular numericalinput feature (e.g., feature value v₃) from the determined positionwithin input dataset 330.

In some instances, executed Shapley-splitter predictive module 350 mayperform operations determine that the Shapley feature value s₃ fails toexceeds the threshold Shapley value s₃* for the additional numericalinput feature (e.g., that s₃≤s₃*), and additionally, or alternatively,that the feature value v₃ of the additional numerical input feature, asmaintained within input dataset 330, fails to satisfy thepredicted-position condition for the additional numerical input featurevalue, as specified within predicted-positive data of the additionalelement. Based on the additional determination, executedShapley-splitter predictive module 350 may establish that the trainedShapley-splitter process is incapable of generating a feature-specificreason for the additional numerical input feature based on thedetermined relationship between the feature values of input dataset 330and the corresponding Shapley feature values. In some examples, executedShapley-splitter predictive module 350 may generate elements of errordata 365 that characterize the determined inability of executedShapley-splitter predictive module to generate a feature-specific reasonassociated with the additional numerical input feature, and route errordata 367 (which includes feature identifier f_(ID,3)) to a local partialdependency plot (PDP) predictive module 366 of executed reasongeneration engine 344.

Executed local PDP predictive module 366 may perform operations thatgenerate a local partial dependency plot associated with the additionalnumerical input feature, and based on the generated partial dependencyplot, determine whether a change in a value of the additional numericalinput feature results in a corresponding increase, or decrease, inpredicted likelihood of the occurrence of the future default eventpredicted by the trained, gradient-boosted, decision-tree process. Basedon the determination, executed local PDP predictive module 366 maygenerate additional, or alternate, elements of textual content thatinclude a feature-specific reason associating a value of the additionalnumerical input feature within with input dataset 330 and thecorresponding element of predicted output data 334, e.g., output dataelement 336. Further, and as described herein, executed local PDPpredictive module 366 may implement any of the local PDP processeddescribed herein concurrently with inferencing by executed predictiveengine 214 (e.g., concurrently with the application of the trained,gradient-boosted, decision-tree process to input datasets 328) andwithout training against one or more of the validation datasets.

By way of example, based on feature identifier f_(ID,3) of theadditional numerical input feature and on the elements of process inputdata 175B, executed local PDP predictive module 366 may performoperations that determine a position of a value of the additionalnumerical input feature within input dataset 330. Further, executedlocal PDP predictive module 366 may also perform operations that, basedon input dataset 330, generate a plurality of modified input datasets368 associated with the additional numerical input feature, and thatprovision each of modified input datasets 368 as an input to executedpredictive engine 214. Executed local PDP predictive module 366 mayestablish a range of feature values associated with, and appropriate to,the additional numerical input feature, and may perform operations thatdiscretize the determined range into discrete intervals (e.g.,consistent with a predetermined number of interpolation points, etc.)and that compute, for each of the discrete intervals, a discretizedfeature value. By way of example, the discretized feature values mayvary linearly across the discretized intervals of the feature range, orin accordance with any additional, or alternate non-linear or linearfunction and in some instances, executed local PDP predictive module 366may perform any of the exemplary processes described herein to generatecorresponding ones of modified input datasets 368 by replacing thatfeature value with a corresponding one of the discretized feature valuesof the additional numerical input feature.

Based on an application of the trained, gradient-boosted decisionprocess the elements of each of modified input datasets 368, executedlocal PDP predictive module 366 may generate one or more elements ofmodified output data 370, and may provision the elements of modifiedoutput data 370 as a further input to executed local PDP predictivemodule 366. As described herein, the local partial dependency plot forthe additional numerical input feature may inspect a marginal effect ofthat additional numerical input feature on the predicted output, andexecuted local PDP predictive module 366 executed local PDP predictivemodule 366 may generate data characterizing the local partial dependencyplot for the additional numerical input feature by averaging thenumerical scores maintained within one or more elements of modifiedoutput data 370 associated with corresponding ones of the discretizedfeature values, and by associated each of the discretized feature valueswith a corresponding one of the average numerical scores (e.g., togenerate corresponding points within the local partial dependency plotfor the additional numerical input feature).

In some examples, executed local PDP predictive module 366 may performoperations that further process the data characterizing the localpartial dependency plot for the additional numerical input feature(e.g., the pairs of discretized feature values and correspondingaveraged numerical scores), and compute a value of a Kendall rankcorrelation coefficient (e.g., a Kendall “τ) for the local partialdependency plot based on the data. If, for example, executed local PDPpredictive module 366 were to establish that the computed value of theKendall rank correlation coefficient exceeds a threshold value, thenexecuted local PDP predictive module 366 may establish that the localpartial dependency plot is characterized by a monotonic increase acrossthe range of feature values of the additional numerical input feature,and may package the phrase “the feature value is too high” into anelement 358C of textual content 358 (e.g., as the correspondingfeature-specific reason).

Alternatively, if executed local PDP predictive module 366 were toestablish that the computed value of the Kendall rank correlationcoefficient fails to exceed the threshold value, then executed local PDPpredictive module 366 may establish that the local partial dependencyplot is characterized by a monotonic decrease across the range offeature values of the additional numerical input feature, and maypackage the phrase “the feature value is too low” into an additionalelement of textual content (not illustrated in FIG. 3B). In additional,or alternate, examples, executed local PDP predictive module 366 mayobtain the feature value of the additional numerical input feature fromthe determined position within input dataset 330, and package theobtained feature value into a further element of textual content, eitherindividually or in conjunction with feature identifier f_(ID,3) (alsonot illustrated in FIG. 3B).

Further, although not illustrated in FIG. 3B, executed Shapley-splitterpredictive module 350 may also establish an inability of the trainedShapley-splitter process to generate a feature-specific reason for anadditional categorical input feature, such as the exemplary categoricalfeatures described herein. Based on elements of error data that includea feature identifier of the additional categorical input feature,executed local PDP predictive module 366 may perform any of theexemplary processes described herein to generate data characterizing alocal partial dependency plot for the additional categorical inputfeature. In some examples, executed local PDP predictive module 366 mayanalyze the data characterizing a local partial dependency plot for theadditional categorical input feature and perform operations thatdetermine a new feature value for the additional categorical inputfeature that would reduce a corresponding, predicted numerical score(e.g., based on the application of the trained, gradient-boosted,decision tree process to an input dataset that includes the new featurevalue). Executed local PDP predictive module 366 may generate anadditional element of textual content that specifies the new featurevalue of the additional categorical input feature, and in someinstances, a feature identifier of the additional categorical inputfeature.

In some examples, executed reason generation engine 344 may perform anyof the exemplary processes described herein to determine afeature-specific reason associated with, and characterizing, each of theadditional, or alternate, ranked pairs of feature identifiers andShapley feature values maintained within extracted subset 348 (based onan application of the exemplary trained Shapley-splitter processes, orthe exemplary local PDP predictive processes, described herein), and togenerate elements of textual content that characterize thefeature-specific reason. As illustrated in FIG. 3B, a reason mappingmodule 374 of executed reason generation engine 344 may receive thetextual content 358, include elements 358A, 358B, and 358C describedherein, along with corresponding ones of the feature identifier,including feature identifiers f_(ID,1), f_(ID,2), and f_(ID,3). Further,in some instances, and executed reason mapping module 374 may performany of the exemplary processes described herein to map thefeature-specific reasons characterized by the elements of textualcontent 358 to corresponding business-specific reasons that, through ause of natural language, characterize the association between thecorresponding feature values and the predicted output in a mannerreadily apparent to, and appreciable by, representatives and customersof the financial institution.

For example, element 358A may be associated with feature identifierf_(ID,1) of the numerical input feature, and may include the phrase “thefeature value is too high.” Further, by way of example, the numericalinput feature may correspond to a current balance associated with acredit-card account held by a customer, and feature identifier f_(ID,1)may include an alphanumeric identifier assigned to the numerical inputfeature by FI computing system 130. In some instances, executed reasonmapping module 374 may obtain elements of mapping data 376 thatassociated feature identifier f_(ID,1) and element 358A (e.g., thefeature-specific reason “the feature value is too low”) with acorresponding feature name (e.g., the feature name “account balance”)and elements of natural language that associate the feature name withthe feature-specific reason (e.g., “account balance is too high”). Basedon the elements of mapping data 376, executed reason mapping module 374may perform operations that package the elements of natural language,either alone or in conjunction with the feature identifier f_(ID,1) intoan element of adverse reason data 378 associated with input dataset 330and output data element 336 (e.g., within element 378A).

Further, element 358B may include feature identifier f_(ID,2) of thecategorical input feature and may include the phrase “the feature valueis not the threshold category.” Further, by way of example, thecategorical input feature may correspond to a current balance associatedwith a past-due interval of a past-due balance associated with thecredit-card, and feature identifier f_(ID,2) may include an alphanumericidentifier assigned to the categorical input feature by FI computingsystem 130. In some instances, the elements of mapping data 376 mayassociate feature identifier f_(ID,2) and element 358B (e.g., thefeature-specific reason “the feature value is not the thresholdcategory”) with a corresponding feature name (e.g., the feature name“past-due interval”) and elements of natural language that associate thefeature name with the feature-specific reason (e.g., “account iscurrently past due”). Based on the elements of mapping data 376,executed reason mapping module 374 may perform operations that packagethe elements of natural language, either alone or in conjunction withthe feature identifier f_(ID,2), into an element of adverse reason data378 (e.g., within element 378B).

Additionally, in some examples, element 358C may include featureidentifier f_(ID,3) of the additional numerical input feature and thephrase “the feature value is too high.” The additional numerical inputfeature may, for example, correspond to the customer's current creditutilization, and feature identifier f_(ID,3) may include an alphanumericidentifier assigned to the categorical input feature by FI computingsystem 130. In some instances, the elements of mapping data 376 mayassociate feature identifier f_(ID,3) and element 358C (e.g., thefeature-specific reason “the feature value is too high”) with acorresponding feature name (e.g., the feature name “credit utilization”)and elements of natural language that associate the feature name withthe feature-specific reason (e.g., “ratio of account utilization ishigh”). Based on the elements of mapping data 376, Executed reasonmapping module 374 may perform operations that package the elements ofnatural language, either alone or in conjunction with the featureidentifier f_(ID,3), into an element of adverse reason data 278 (e.g.,within element 3780).

In some instances, executed reason generation engine 344 may perform anyof the exemplary processes described herein to map the feature-specificreasons characterized by the each, or a selected subset, of the elementsof textual content 358 (e.g., elements 358A, 358B, and 358C) tocorresponding elements of natural language, and to generate anadditional, or alternate, element of adverse reason data 378 thatincludes the mapped elements of nature language and a correspondingfeature identifier. The selected subset of the elements of textualcontent 358 may, for example, include a predetermined number of elementsof textual content 358, which may be associated with correspondingnumerical or categorical features characterized by the largest Shapleyfeature values (e.g. as specified within subset 348 of the ranked pairsof feature identifiers and Shapley feature values). Executed reasongeneration engine 344 may also provision the elements of adverse reasondata 378 associated with associated with input dataset 330 and withoutput data element 336, including elements 378A, 378B, and 378C, asadditional inputs to executed post-processing engine 338.

Further, although not illustrated in FIG. 3B, executed reason generationengine 344 may also perform any of the exemplary processes describedherein to generate elements of textual content that identify andcharacterize a feature-specific reason associated with all, or aselected subset, of the input feature values maintained within eachadditional, or alternate, input datasets 328 and as such, withcorresponding elements of predicted output data 334, and to map thefeature-specific reasons characterized by the textual content tocorresponding elements of natural language, and to generate elements ofadverse reason data that includes the mapped elements of nature languageand a corresponding feature identifier. In some instances, executedreason generation engine 344 may provision the additional elements ofadverse reason data associated with each of the additional, oralternate, ones of input datasets 328, and with each of the additional,or alternate, elements of predicted output data 334, as further inputsto executed post-processing engine 338.

As described herein, executed post-processing engine 338 may receive thegenerated elements of output data 334 (e.g., either alone, or inconjunction with corresponding ones of input datasets 328) from executedpredictive engine 214, and may receive the elements of adverse reasondata 378 (e.g., including elements 378A, 378B, and 378C associated withinput dataset 330 and with output data element 336) from executed reasongeneration engine 344. In some instances, executed post-processingengine 338 may perform operations that access the elements of customerdata 302 maintained within aggregated data store 132, and associate eachof the elements of customer data 302 (e.g., that identify acorresponding one of the customers of the financial institution thathold an unsecured credit product) with a corresponding one of theelements of output data 334 (e.g., that include numerical scoresindicative of the predicted likelihood that corresponding ones of thecustomers will be involved in a default event during the future temporalinterval) and with a corresponding subset of the elements of adversereason data 378 (that include elements of natural language characterizethe adverse reasons for decisions associated with the numerical scores).

By way of example, element 336 of output data 334 may be associated withthe particular customer identified by element 306 of customer data 302,and may include a numerical score of unity, which indicates a predictedoccurrence of a default event involving the particular customer duringthe future temporal interval. Further, elements 378A, 378B, and 378C ofadverse reason data 378 may also be associated with the particularcustomer, and may include the elements of natural languagecharacterizing, and specifying, the adverse reasons for the predictedscore of unity that include, but are not limited to, respective ones of“account balance is too high,” “account is currently past due,” and“ratio of account utilization is high.” Executed post-processing engine338 may, in some instances, associate customer identifier 308 withelement 336 of output data 334 and with elements 378A, 378B, and 378C ofadverse reason data 378, and may perform any of these exemplaryprocesses to associate each additional, or alternate, one of theelements of output data 334 and adverse reason data 378 with acorresponding one of the customer identifiers maintained within customerdata 302.

Further, and in some instances, executed post-processing engine 338 mayperform operations that sort the associated elements of customer data302, output data 334, and adverse reason data 378 based on thecorresponding numerical scores (e.g., which indicate the predictedlikelihood that corresponding ones of the customer will be involved in adefault event during the future temporal interval)), and output elementsof sorted output data 380 that include the associated, and now sorted,elements of customer data 302, output data 334, and adverse reason data378. For example, and for the particular customer, sorted output data380 may include a corresponding sorted element 382 that associatestogether customer identifier 308, element 336 of output data 334 (e.g.,that specifies a numerical score of unity for the particular customer),and the subset of the elements of adverse reason data 378 (e.g.,elements 378A, 378B, and 378C that specify, in natural language, theadverse reasons for the numerical score of unity). As illustrated inFIG. 3B, FI computing system 130 may perform operations that transmitall, or a selected portion of, sorted output data 380 to issuer system301 (and additionally, or alternatively, to other ones of the issuersystems).

Referring to FIG. 3C, issuer system 301 may receive, all, or a selectedportion, of sorted output data 380 from FI computing system 130. Forexample, a programmatic interface associated with and maintained byissuer system 301, such as application programming interface (API) 382,may receive and route sorted output data 380 to a credit modificationengine 384 executed by the one or more processors of issuer system 301.As described herein, the elements of sorted output data 380 mayassociate together customer identifiers (e.g., that identifying andcharacterize corresponding customer of the financial institution),corresponding elements of output data 334 (which include numericalscores indicative of a predicted likelihood that the corresponding onesof the customers will be involved in a default event during the futuretemporal interval), and corresponding subsets of the elements of adversereason data 378 (which specify the adverse reason(s) for each of thepredicted likelihoods), and may sort (or group) the associated customeridentifiers, corresponding elements of output data 334, andcorresponding subsets of the elements of adverse reason data 378 intorespective bins indicative of a predicted non-occurrence of the defaultevent during the future temporal interval (e.g., associated with anumerical score of zero), and indicative of a predicted occurrence ofthe default event during the future temporal interval (e.g., associatedwith a numerical score of unity).

For example, the customer of the financial institution that requestedthe credit-limit increase may be associated with customer identifier 308and as such, with sorted element 380 that associates together customeridentifier 308, and element 336 of output data 334 (which specifies anumerical score of unity for the customer), and elements 378A, 378B, and378C of adverse reason data 378 (which specify, as adverse reasons forthe numerical score of unity, elements of natural language “accountbalance is too high,” “account is currently past due,” and “ratio ofaccount utilization is high”). Executed credit modification engine 384may also access modification criterion 386, which may specify amodification threshold for increasing the credit limit of thecredit-card account, and based on modification criterion 386, determinethat the numerical value of unity exceeds the modification threshold.Based on the determination that the numerical value of unity exceeds themodification threshold, FI computing system 130 may decline to increasethe credit limit of the customer's credit-card account, and executedcredit modification engine 384 may generate elements of notificationdata 388 that confirm the decision to decline the requested credit-limitincrease, and that include each of elements 378A, 378B, and 378C ofadverse reason data 378, which specify the adverse reasons for thedeclined credit-limit increase. Executed credit modification engine 384may also perform operations that transmit notification data 388 acrossnetwork 120 to a computing system or system associated with thecustomer, such as customer device 390.

In some instances, not illustrated in FIG. 3C, one or more applicationprograms executed by customer device 390, such as the executed webbrowser or executed mobile banking application, may receive the elementsof notification data 388 through a corresponding programmatic interface.The one or more executed application programs may process the elementsof notification data 388, and render all, or a selected portion of,these elements within a digital interface, e.g., via a correspondingdisplay unit. As illustrated in FIG. 3C, the digital interface mayinclude interface elements 392 that confirm the decision to decline therequested credit-limit increase, and that identify the adverse reasonsthat drive the decision, such, as but not limited to, “account balanceis too high,” “account is currently past due,” and “ratio of accountutilization is high.”

FIG. 4 is a flowchart of an exemplary process 400 for adaptivelytraining a machine learning or artificial intelligence process topredict a likelihood of an occurrence of an event during a futuretemporal interval using training datasets associated with a first priortemporal interval, and using validation datasets associated with asecond, and distinct, prior temporal interval. As described herein, themachine-learning or artificial-intelligence process may include anensemble or decision-tree process, such as a gradient-boosteddecision-tree process (e.g., the XGBoost process), the event mayinclude, but is not limited to, a default event involving a customer ofa financial institution and corresponding credit product, such as asecured or unsecured credit-card account, and the future temporalinterval may include a twelve-month interval disposed subsequent to atemporal prediction point. In some instances, one or more computingsystems, such as, but not limited to, one or more of the distributedcomponents of FI computing system 130, may perform one or of the stepsof exemplary process 300, as described herein.

Referring to FIG. 4 , FI computing system 130 may perform any of theexemplary processes described herein to establish a secure, programmaticchannel of communication with one or more additional computing systems,such as source systems 102A and 102B and transaction system 110 of FIG.1A, and to obtain, from the source computing systems, elements ofinteraction data that identify and characterize one or more customers ofthe financial institution (e.g., in step 402 of FIG. 4 ). The elementsof interaction data may include, but are not limited to, one or moreelements of customer profile data, account data, transaction data,delinquency data, and/or credit-bureau data associated withcorresponding ones of the customers, and FI computing system 130 mayalso perform operations that store (or ingest) the obtained elements ofinteraction data within one or more accessible data repositories, suchas aggregated data store 132 (e.g., also in step 402 of FIG. 4 ). Insome instances, FI computing system 130 may perform the exemplaryprocesses described herein to obtain and ingest the elements ofinteraction data in accordance with a predetermined temporal schedule(e.g., on a monthly basis), or a continuous streaming basis, across thesecure, programmatic channel of communication.

Further, FI computing system 130 may access the ingested elements ofinternal and external interaction data, and may perform any of theexemplary processes described herein to pre-process the ingestedelements of internal and external interaction data elements (e.g., theelements of customer profile, account, transaction, delinquency, and/orcredit bureau data described herein) and generate one or moreconsolidated data records (e.g., in step 404 of FIG. 4 ). As describedherein, the FI computing system 130 may store each of the consolidateddata records within one or more accessible data repositories, such asconsolidated data store 144 (e.g., also in step 404 of FIG. 4 ).

For example, and as described herein, each of the consolidated datarecords may be associated with a particular one of the customers, andmay include a corresponding pair of a customer identifier associatedwith the particular customer (e.g., an alphanumeric character string,etc.) and a temporal interval that identifies a corresponding temporalinterval. Further, and in addition to the corresponding pair of customerand temporal identifiers, each of the consolidated data records may alsoinclude one or more consolidated elements of customer profile, account,transaction, delinquency, and/or credit bureau data that characterizethe particular customer during the corresponding temporal intervalassociated with the temporal identifier, along one or more aggregatedvalues of customer profile, account, delinquency, credit-bureau, and/ortransaction parameters that characterize a behavior of the particularcustomer during the corresponding temporal interval.

In some instances, FI computing system 130 may perform any of theexemplary processes described herein to decompose the consolidated datarecords into (i) a first subset of the consolidated data records havingtemporal identifiers associated with a first prior temporal interval(e.g., the training interval Δt_(training), as described herein) and(ii) a second subset of the consolidated data records having temporalidentifiers associated with a second prior temporal interval (e.g., thevalidation interval Δt_(validation), as described herein), which may beseparate, distinct, and disjoint from the first prior temporal interval(e.g., in step 406 of FIG. 4 ). By way of example, portions of theconsolidated data records within the first subset may be appropriate totrain the machine-leaning or artificial process (e.g., thegradient-boosted, decision-tree process described herein) during thetraining interval Δt_(training), and portions of the consolidatedrecords within the second subset may be appropriate to validating thetrained, gradient-boosted, decision-tree process during the validationinterval Δt_(validation). FI computing system 130 may also perform anyof the exemplary processes described herein to filter the consolidateddata records of the first and second subsets in accordance with one ormore filtration criteria, such as the exemplary filtration criteriadescribed herein (e.g., in step 408 of FIG. 4 ).

In some instances, FI computing system 130 may perform any of theexemplary processes described herein to generate a plurality of trainingdatasets based on elements of data obtained, extracted, or derived fromall or a selected portion of the first subset of the consolidated datarecords (e.g., in step 410 of FIG. 4 ). By way of example, each of theplurality of training datasets may be associated with a correspondingone of the customers of the financial institution and a correspondingtemporal interval, and may include, among other things a customeridentifier associated with that corresponding customer and a temporalidentifier representative of the corresponding temporal interval, asdescribed herein. Further, and as described herein, each of theplurality of training datasets may also elements of data (e.g., featurevalues) that characterize the corresponding one of the customers, thecorresponding customer's interaction with the financial institution orwith other financial institution, and/or an occurrence (or lack thereof)of default events involving the corresponding customer during a temporalinterval disposed prior to the corresponding temporal interval, e.g.,during the extraction interval Δt_(extract) described herein. Further,each of the plurality of training datasets may also include an elementof ground-truth data indicative of the presence or absence of an actualdefault event associated with a corresponding one of the customerswithin a corresponding target prediction interval Δt_(target), such as,but not limited to, a twelve-month period disposed subsequent to thedate specified by the temporal identifier).

Based on the plurality of training datasets, FI computing system 130 mayalso perform any of the exemplary processes described herein to trainthe machine-learning or artificial-intelligence process (e.g., thegradient-boosted decision-tree process described herein) to predict,during a current temporal interval, a likelihood of occurrences ofdefault events involving customers of the financial institution during afuture temporal interval (e.g., in step 412 of FIG. 4 ). For example,and as described herein, FI computing system 130 may perform operationsthat establish a plurality of nodes and a plurality of decision treesfor the gradient-boosted, decision-tree process, which may ingest andprocess the elements of training data (e.g., the customer identifiers,the temporal identifiers, the feature values, etc.) maintained withineach of the plurality of training datasets, and that train thegradient-boosted, decision-tree process against the elements of trainingdata included within each of the plurality of the training datasets.

In some examples, the distributed components of FI computing system 130may perform any of the exemplary processes described herein in parallelto establish the plurality of nodes and a plurality of decision treesfor the gradient-boosted, decision-tree process, and to train thegradient-boosted, decision-tree process against the elements of trainingdata included within each of the plurality of the training datasets. Theparallel implementation of these exemplary training processes by thedistributed components of FI computing system 130 may, in someinstances, be based on an implementation, across the distributedcomponents, of one or more of the parallelized, fault-tolerantdistributed computing and analytical protocols described herein.

Through the performance of these training processes, FI computing system130 may compute one or more candidate process parameters thatcharacterize the trained machine-learning or artificial-intelligenceprocess, such as, but not limited to, candidate process parameters forthe trained, gradient-boosted, decision-tree process described herein,such as, but not limited to, the exemplary process parameters describedherein (e.g., in step 414 of FIG. 4 ). Further, and based on theperformance of these training processes, FI computing system 130 mayperform any of the exemplary processes described herein to generatecandidate input data, which specifies a candidate composition of aninput dataset for the trained machine-learning or artificialintelligence process, such as the trained, gradient-boosted,decision-tree process described herein (e.g., also in step 414 of FIG. 4).

Further, FI computing system 130 may perform any of the exemplaryprocesses described herein to access the second subset of theconsolidated data records, and to generate a plurality of validationsubsets having compositions consistent with the candidate input data(e.g., in step 416 of FIG. 4 ). As described herein, each of theplurality of the validation datasets may be associated with acorresponding one of the customers of the financial institution, andwith a corresponding temporal interval within the validation intervalΔt_(validation), and may include a customer identifier associated withthe corresponding one of the customers and a temporal identifier thatidentifies the corresponding temporal interval. Further, each of theplurality of the validation datasets may also include one or morefeature values consistent with the candidate input data, associated withthe corresponding one of the customers, and obtained, extracted, orderived from corresponding ones of the accessed second subset of theconsolidated data records (e.g., during the corresponding extractioninterval Δt_(extract), as described herein).

In some instances, FI computing system 130 may perform any of theexemplary processes described herein to apply the trainedmachine-learning or artificial intelligence process (e.g., the trained,gradient-boosted, decision-tree process described herein) to respectiveones of the validation datasets, and to generate corresponding elementsof output data based on the application of the trained machine-learningor artificial intelligence process to the respective ones of thevalidation datasets (e.g., in step 418 of FIG. 4 ). As described herein,each of the generated elements of output data may be associated with acorresponding one of the validation datasets and as such, acorresponding one of the customers. Further, each of the generatedelements of output data may also a numerical score (e.g., ranging fromzero to unity) indicative of a predicted likelihood that thecorresponding one of the customers will experience, or will be involvedin, a default event within a future temporal interval, such as, but notlimited to, a twelve-month interval disposed subsequent to the datespecified by the temporal identifier within the respective one of thevalidation datasets.

Further, and as described herein, the distributed components of FIcomputing system 130 may perform any of the exemplary processesdescribed herein in parallel to validate the trained, gradient-boosted,decision-tree process described herein based on the application of thetrained, gradient-boosted, decision-tree process (e.g., configured inaccordance with the candidate process parameters) to each of thevalidation datasets. The parallel implementation of these exemplaryvalidation processes by FI computing system 130 may, in some instances,be based on an implementation, across the distributed components, of oneor more of the parallelized, fault-tolerant distributed computing andanalytical protocols described herein.

In some examples, FI computing system 130 may perform any of theexemplary processes described herein to compute a value of one or moremetrics that characterize a predictive capability, and an accuracy, ofthe trained machine-learning or artificial intelligence process (such asthe trained, gradient-boosted, decision-tree process described herein)based on the generated elements of output data and corresponding ones ofthe validation datasets (e.g., in step 420 of FIG. 4 ), and to determinewhether all, or a selected portion of, the computed metric valuessatisfy one or more threshold conditions for a deployment of the trainedmachine-learning or artificial intelligence process (e.g., in step 422of FIG. 4 ). As described herein, and for the trained, gradient-boosted,decision-tree process, the computed metrics may include, but are notlimited to, one or more recall-based values (e.g., “recall@5,”“recall@10,” “recall@20,” etc.), one or more precision-based values forthe trained, gradient-boosted, decision-tree process, and additionally,or alternatively, a computed value of an area under curve (AUC) for aprecision-recall (PR) curve or a computed value of an AUC for a receiveroperating characteristic (ROC) curve associated with the trained,gradient-boosted, decision-tree process.

Further, and as described herein, the threshold requirements for thetrained, gradient-boosted, decision-tree process may specify one or morepredetermined threshold values, such as, but not limited to, apredetermined threshold value for the computed recall-based values, apredetermined threshold value for the computed precision-based values,and/or a predetermined threshold value for the computed AUC values. Insome examples, FI computing system 130 may perform any of the exemplaryprocesses described herein to establish whether one, or more, of thecomputed recall-based values, the computed precision-based values, orthe computed AUC values exceed, or fall below, a corresponding one ofthe predetermined threshold values and as such, whether the trained,gradient-boosted, decision-tree process satisfies the one or morethreshold requirements for deployment.

If, for example, FI computing system 130 were to establish that one, ormore, of the computed metric values fail to satisfy at least one of thethreshold requirements (e.g., step 422; NO), FI computing system 130 mayestablish that the trained machine-learning or artificial-intelligenceprocess (e.g., the trained, gradient-boosted, decision-tree process) isinsufficiently accurate for deployment and a real-time application tothe elements of customer profile, account, transaction, delinquency, orcredit-bureau data described herein. Exemplary process 400 may, forexample, pass back to step 410, and FI computing system 130 may performany of the exemplary processes described herein to generate additionaltraining datasets based on the elements of the consolidated data recordsmaintained within the first subset.

Alternatively, if FI computing system 130 were to establish that eachcomputed metric value satisfies threshold requirements (e.g., step 422;YES), FI computing system 130 may deem the trained machine-learning orartificial intelligence process (e.g., the trained gradient-boosted,decision-tree process described herein) ready for deployment andreal-time application to the elements of customer profile, account,transaction, delinquency, or credit-bureau data described herein, andmay perform any of the exemplary processes described herein to generateprocess parameter data that includes the candidate process parameters,and process input data that includes the candidate input data,associated with the of the trained machine-learning or artificialintelligence process (e.g., in step 424 of FIG. 4 ). Exemplary process400 is then complete in step 426.

FIG. 5 is a flowchart of an exemplary process 500 for training anexplainability process based on validation data associated with atrained, gradient-boosted decision-tree process. As described herein,the explainability process may include, but is not limited to,Shapley-splitter process that, when applied to a value of a numerical orcategorical feature of a customer-specific dataset, generates elementsof textual content that link together the numerical or categoricalfeature value to the predicted output generated through the applicationof the trained gradient-boosted, decision-tree process to thecustomer-specific input dataset, and that establish a feature-specificreason for the predicted output, such as, but not limited to, apredicted likelihood of an occurrence of a customer-specific defaultevent during a future temporal interval, as described herein. In someinstances, one or more computing systems, such as, but not limited to,one or more of the distributed components of FI computing system 130,may perform one or of the steps of exemplary process 500, as describedherein.

Referring to FIG. 5 , FI computing system 130 may perform any of theexemplary processes described herein to obtain elements of process inputdata associated with trained, gradient-boosted decision tree process,validation datasets associated with the trained, gradient-boosteddecision tree process, and elements of validation output data generatedthrough an application of the trained, gradient-boosted decision treeprocess to the elements of corresponding ones of the validation datasets(e.g., in step 502 of FIG. 5 ). As described herein, the elements ofprocess input data may characterize a composition of an input datasetfor the trained, gradient-boosted, decision-tree process and may includefeature identifiers of corresponding ones of the numerical orcategorical input feature included within the input dataset, andfurther, each of the validation datasets may be structured in accordancewith the elements of process input data.

In step 504 of FIG. 5 , FI computing system 130 may perform any of theexemplary processes described herein to compute, based on the validationdatasets and the elements of output data, a Shapley feature value that,for each of the numerical or categorical input feature values includedwithin one or more of the validation datasets, characterizes acontribution of the numerical or categorical input feature value to thepredicted output of the trained, gradient-boosted, decision-treeprocess, e.g., the predicted likelihood of an occurrence of a defaultinvolving a corresponding customer and a credit-card account during afuture temporal interval. FI computing system 130 may also perform anyof the exemplary processes described herein to generate a plurality oftraining samples based on the computed Shapley feature values (e.g., instep 506 of FIG. 5 ). For example, and as described herein, each of thetraining samples may a feature identifier of a corresponding one of thenumerical or categorical input features (e.g., obtained from theelements of process input data), a value of the corresponding one of thenumerical or categorical input features (e.g., obtained from thevalidation datasets), and a corresponding one of the computed Shapleyfeature values.

FI computing system 130 may also perform any of the exemplary processesdescribed herein to select one of the numerical or categorical inputfeatures for training (e.g., in step 508 of FIG. 5 ), and that obtain afeature-specific subset of the training samples associated with theselected one of the numerical or categorical input features (e.g., instep 510 of FIG. 5 ). Further, in step 510, FI computing system 130 mayalso perform any of the exemplary processes described herein to sort thetraining samples within the obtained feature-specific subset inaccordance with the Shapley feature values (e.g., in descending orderbased on the corresponding Shapley feature values, etc.).

In some examples, FI computing system 130 may perform operations thatdetermine whether the selected one of the numerical or categorical inputfeatures corresponds to a numerical input feature (e.g., in step 512 ofFIG. 5 ). If FI computing system 130 were to establish that the selectedone of the numerical or categorical input features corresponds to anumerical input feature (e.g., in step 512; YES), FI computing system130 may perform any of the exemplary processes described herein todetermine, for the numerical feature value, a threshold feature value v*and a threshold Shapley value s* that maximize values of precision andrecall for the selected numerical input feature and as such, thatmaximize a F₁ score associated with the selected numerical feature(e.g., in step 514 of FIG. 5 ). By way of example, and through animplementation of one or more of the exemplary processes describedherein in step 514, FI computing system 130 may compute the thresholdfeature value v* and the threshold Shapley value s* for the selectednumerical input feature in accordance with:

${\left( {v^{*},s^{*}} \right) = {\underset{{({v,s})} \in R^{2}}{\arg\max}\frac{2{B\left( {v,s} \right)}}{{2{B\left( {v,s} \right)}} + {A\left( {v,s} \right)} + {D\left( {v,s} \right)}}}},$

where: A(v,s) corresponds to a number of the sorted training sampleswithin the obtained feature-specific subset having a numerical featurevalue that fails to exceed the threshold feature value v* and a Shapleyfeature value that exceeds the threshold Shapley value s*; B(v,s)corresponds to a number of the sorted training samples within theobtained feature-specific subset having a numerical feature value thatexceeds the threshold feature value v* and a Shapley feature value thatexceeds the threshold Shapley value s*; and D(v,s) corresponds to anumber of the sorted training samples within the obtainedfeature-specific subset having a numerical feature value that exceedsthe threshold feature value v* and a Shapley feature value that failsexceeds the threshold Shapley value s*.

FI computing system 130 may also perform operations, described herein,that package the feature identifier of the selected numerical inputfeature, threshold data that specifies the threshold feature value v*and the threshold Shapley value s* for the numerical input feature,elements of predicted-positive data associated with the selectednumerical input feature into an element of numerical feature parameterdata (e.g., in step 516 of FIG. 5 ). The elements of predicted-positivedata may, for example, characterize an occurrence of a predictedpositive based on the application of the trained, gradient-boosted,decision-tree process to a value of the selected numerical inputfeature, and may specify elements of textual content (e.g., afeature-specific reason) that characterize a causal relationship betweenthe value of the selected numerical feature and the occurrence of thepredicted positive. Further, in step 516, FI computing system may storethe element of numerical feature parameter data within a datarepository.

In some instances, FI computing system 130 may parse the elements ofprocess input data and determine whether additional numerical orcategorical features await selection for training (e.g., in step 518 ofFIG. 5 ). If, for example, FI computing system 130 were to determinethat no further additional numerical or categorical features awaitselection for training (e.g., step 518; NO), exemplary process 500 maybe complete in step 520. Alternatively, if FI computing system 130 wereto determine that additional numerical or categorical features awaitselection for training (e.g., step 518; YES), exemplary process 500 maypass back to step 508, and FI computing system 130 may performoperations to select an additional one of the numerical or categoricalinput features for training.

Further, referring back to step 512, If FI computing system 130 were toestablish that the selected one of the numerical or categorical inputfeatures corresponds to a categorical input feature (e.g., in step 512;NO). FI computing system 130 may perform any of the exemplary processesdescribed obtain data that identifies a plurality of candidatecategories associated with the selected categorical input feature, whichinclude, in some instances, a null value indicating an absence of acategory (e.g., in step 522 of FIG. 5 ). FI computing system 130 mayalso perform any of the exemplary processes described herein todetermine a threshold category c* and a threshold Shapley value s* forthe selected categorical input feature that maximize values of precisionand recall for the selected categorical input feature and as such, thatmaximize an F₁ score associated with the selected categorical inputfeature (e.g., in step 524 of FIG. 5 ). By way of example, and throughan implementation of one or more of the exemplary processes describedherein in step 524, FI computing system 130 may compute the thresholdcategory c* and the threshold Shapley value s* for the selectedcategorical input feature in accordance with:

${\left( {c^{*},s^{*}} \right) = {\underset{{({c,s})} \in R^{2}}{\arg\max}\frac{2{B\left( {c,s} \right)}}{{2{B\left( {c,s} \right)}} + {A\left( {c,s} \right)} + {D\left( {c,s} \right)}}}},$

where: A(c,s) corresponds to a number of the sorted training sampleswithin the obtained feature-specific subset having a categorical featurevalue inconsistent with the threshold category c* and a Shapley featurevalue that exceeds the threshold Shapley value s*; B(v,s) corresponds toa number of the sorted training samples within the obtainedfeature-specific subset having a categorical feature value that isconsistent with the threshold category c* and a Shapley feature valuethat exceeds the threshold Shapley value s*; and D(v,s) corresponds to anumber of the sorted training samples within the obtainedfeature-specific subset having a categorical feature value that isconsistent with the threshold category c* and a Shapley feature valuethat fails to exceeds the threshold Shapley value s*.

FI computing system 130 may also perform operations, described herein,that package the feature identifier of the selected categorical inputfeature, threshold data that specifies the threshold category c* and thethreshold Shapley value s*, and elements of predicted-positive dataassociated with the selected categorical input feature into an elementof categorical feature parameter data (e.g., in step 526 of FIG. 5 ).The elements of predicted-positive data may, for example, characterizean occurrence of a predicted positive based on the application of thetrained, gradient-boosted, decision-tree process to a value of theselected categorical input feature, and may specify elements of textualcontent (e.g., a feature-specific reason) that characterize a causalrelationship between the value of the selected categorical input featureand the occurrence of the predicted positive. Further, in step 526, FIcomputing system may store the element of numerical feature parameterdata within a data repository.

Exemplary process 500 may then pass back to step 518, and FI computingsystem 130 may parse the elements of process input data and determinewhether additional numerical or categorical features await selection fortraining

FIG. 6A is a flowchart of an exemplary process 600 for generatingelements of adverse reason data characterizing a predicted output of atrained machine-learning or artificial-intelligence process based on anapplication of one or more explainability processes to customer-specificinput datasets, in accordance with the disclosed exemplary embodiments.As described herein, the events may include one or more default eventsinvolving corresponding ones of the customers, and the machine-learningor artificial-intelligence process may include an ensemble ordecision-tree process, such as a gradient-boosted decision-tree process(e.g., the XGBoost process), which may be trained to predict alikelihood of an occurrence of a default event during a future temporalinterval using training datasets associated with a first prior temporalinterval (e.g., the training interval Δt_(training), as describedherein), and using validation datasets associated with a second, anddistinct, prior temporal interval (e.g., the validation intervalΔt_(validation), as described herein). In some instances, one or morecomputing systems, such as, but not limited to, one or more of thedistributed components of FI computing system 130, may perform one or ofthe steps of exemplary process 600, as described herein.

Referring to FIG. 6A, FI computing system 130 may perform any of theexemplary processes described herein to receive elements of customerdata associated with a customer of the financial institution (e.g., instep 602 of FIG. 6A). For example, FI computing system 130 may receivethe elements of customer data from one or more additional computingsystems associated with, or operated by, the financial institution (suchas, but not limited to, issuer system 301), and in some instances, FIcomputing system 130 may perform any of the exemplary processesdescribed herein to store the obtained elements of customer data withina locally accessible data repository (e.g., within aggregated data store132). Further, in some instances, FI computing system 130 may alsoperform any of the exemplary processes described herein to synchronizeand merge the obtained elements of customer data with one or morepreviously ingested elements of customer data maintained within thelocally accessible data repository. As described herein, the elements ofcustomer data may include a customer identifier associated with thecustomer (e.g., the alphanumeric character string, etc.) and a systemidentifier associated with a corresponding one of the additionalcomputing systems (e.g., an IP or MAC address of issuer system 301,etc.).

FI computing system 130 may also perform any of the exemplary processesdescribed herein to obtain elements of process parameter data thatspecify one or more process parameters for the trained,gradient-boosted, decision-tree process, such as the exemplary processparameters described herein, and to obtain element of process input datathat specify a composition of an input dataset for the trained,gradient-boosted, decision-tree process (e.g., in step 604 of FIG. 6A).As described herein the elements of process input data may specify thecomposition of the input dataset for the trained, gradient-boosted,decision-tree process, which not only includes feature identifiersassociated with each of the numerical or categorical input featurevalues, but also a specified sequence or position of these numerical orcategorical input feature values within the input dataset.

In some instances, FI computing system 130 may access the elements ofcustomer data, and may perform any of the exemplary processes describedherein to generate a customer-specific input dataset having acomposition consistent with the elements of process input data (e.g., instep 606 of FIG. 6A). Further, and based on the one or more obtainedprocess parameters, FI computing system 130 may perform any of theexemplary processes described herein to apply the trained,gradient-boosted, decision-tree process to the generated,customer-specific input dataset (e.g., in step 608 of FIG. 6A), and togenerate a customer-specific element of predicted output data associatedwith the customer-specific input dataset (e.g., in step 610 of FIG. 6A).For example, and based on the one or more obtained process parameters,FI computing system 130 may perform operations, described herein, thatestablish a plurality of nodes and a plurality of decision trees for thetrained, gradient-boosted, decision-tree process, each of which receive,as inputs (e.g., “ingest”), corresponding elements (e.g., numerical orcategorical input feature values) of the customer-specific inputdataset.

Based on the ingestion of the input datasets by the established nodesand decision trees of the trained, gradient-boosted, decision-treeprocess, FI computing system 130 may perform operations that apply thetrained, gradient-boosted, decision-tree process to thecustomer-specific input dataset and that generate the customer-specificelement of the output data associated with the customer-specific inputdataset. As described herein, the customer-specific element of predictedoutput data may include a numerical score (e.g., either zero or unity)indicative of a predicted likelihood of an occurrence of a default eventinvolving the customer and a corresponding credit-card account duringthe future temporal interval, e.g., with a score of zero beingindicative of a predicted non-occurrence of the default event during thefuture temporal interval, and with a score of unity being indicative ofa predicted occurrence of the default event during the future temporalinterval. Further, and as described herein, the future temporal intervalmay include, but is not limited to, a twelve-month period disposedsubsequent to a corresponding prediction date (e.g., the prediction datet_(pred) described herein).

Further, as illustrated in FIG. 6A, FI computing system 130 may alsoperform any of the exemplary processes described herein to generateelements of adverse reason data that include elements of naturallanguage explaining a magnitude of the numerical score generated throughthe application of the trained, gradient-boosted, decision-tree processto the generated, customer-specific input dataset (e.g., in step 612 ofFIG. 6A). By way of example, in step 612, FI computing system 130 mayperform any of the exemplary processes described herein to apply one ormore explainability processes to the numerical or categorical featurevalues of the customer-specific input dataset, to generate elements oftextual content that establish a causal relationship betweencorresponding ones of the numerical or categorical feature values andthe predicted output of the trained, gradient-boosted, decision-treeprocess, and to map the elements of textual content to correspondingelements of natural language that characterize the association betweenthe corresponding numerical or categorical feature values and thepredicted output in a manner readily apparent to, and appreciable by,representatives and customers of the financial institution, as describedbelow in reference to FIG. 6B.

FIG. 6B is a flowchart of an exemplary process 650 for applying one ormore explainability processes to an input dataset associated with atrained machine-learning or artificial-intelligence process, accordingto some exemplary embodiments. As described herein, the predicted outputmay include a customer-specific numerical score (e.g., either zero orunity) indicative of a predicted likelihood of an occurrence of adefault event involving a customer and a corresponding credit-cardaccount during a future temporal interval, e.g., with a score of zerobeing indicative of a predicted non-occurrence of the default eventduring the future temporal interval, and with a score of unity beingindicative of a predicted occurrence of the default event during thefuture temporal interval. Further, and as described herein, the futuretemporal interval may include, but is not limited to, a twelve-monthperiod disposed subsequent to a corresponding prediction date (e.g., thetemporal prediction point t_(pred) described herein). In some instances,one or more computing systems, such as, but not limited to, one or moreof the distributed components of FI computing system 130, may performone or of the steps of exemplary process 650, as described herein.

Referring to FIG. 6B, FI computing system 130 may obtaincustomer-specific input dataset associated with the trained,gradient-boosted, decision tree process and a customer-specific elementof predicted output data generated through an application of thetrained, gradient-boosted, decision tree process to thecustomer-specific input dataset (e.g., in step 652 of FIG. 6B). Based onthe customer-specific input dataset and element of predicted outputdata, FI computing system 130 may perform any of the exemplary processesdescribed herein to compute a Shapley feature value for each of thenumerical or categorical input features included within thecustomer-specific input dataset (e.g., in step 654 of FIG. 6B). Further,FI computing system 130 may also perform any of the exemplary processesdescribed herein to associate each of computed Shapley feature valueswith an identifier of the corresponding numerical or categorical inputfeature, and to rank each of the associated pairs feature identifiersand Shapley feature values based on corresponding ones of Shapleyfeature values, for example, in descending order (e.g., in step 656 ofFIG. 6B). FI computing system 130 may also perform any of the exemplaryprocesses described herein to identify a subset of the ranked pairs offeature identifiers and Shapley feature values associated with thosenumerical or categorical input features characterized by the largest ofthe ranked Shapley feature values (e.g., in step 658 of FIG. 6B). Forexample, the subset may include a predetermined number of the rankedpairs associated with the largest of the ranked Shapley feature values,or the subset may include a plurality of ranked pairs associated withShapley feature values that exceed a threshold value.

In some instances, FI computing system 130 may perform operation,described herein, to select a corresponding one of the ranked pairs offeature identifiers and Shapley feature values (e.g., in step 660 ofFIG. 6B). Further, FI computing system 130 may also perform operations,described herein, that obtain the feature identifier and the Shapleyfeature value from the selected ranked pair, and that access the featurevalue associated with the obtained feature identifier from thecustomer-specific input dataset (e.g., also in step 660 of FIG. 6B).

Based on the feature identifier obtained from the selected ranked pair,FI computing system 130 may perform any of the exemplary processesdescribed herein to obtain an element of numerical or categoricalfeature parameter data associated with a rained Shapley-splitter processthat includes the obtained feature identifier, and as such, thecharacterizes an application of the trained Shapley-splitter process toobtained feature value (e.g., in step 662 of FIG. 6B). By way ofexample, the obtained element of numerical or categorical featureparameter data may include, among other things, a corresponding featureidentifier, a corresponding threshold feature value or thresholdcategory, a corresponding threshold Shapley value, and elements ofpredicted-positive data specifying a predicted-positive condition forthe numerical or categorical input feature associated with the selectedranked pair. FI computing system 130 may also perform operations thatdetermine whether the Shapley feature value of the selected ranked pairexceeds the corresponding threshold Shapley value, and whether theaccessed feature value satisfies the predicted-positive condition setforth in the corresponding elements of predicted-positive data (e.g., instep 664 of FIG. 6B).

If, for example, FI computing system 130 were to determine that theShapley feature value of the selected ranked pair exceeds thecorresponding threshold Shapley value, and that the accessed featurevalue satisfies the predicted-positive condition (e.g., step 664; YES),FI computing system 130 may perform any of the exemplary processesdescribed herein to apply the trained Shapley-splitter process to theaccessed feature value and based on the application of the trainedShapley-splitter process to the accessed feature value, generateelements of textual content that establish a causal relationship betweenthe accessed feature value and the predicted output of the trained,gradient-boosted, decision-tree process, and as such, establish afeature-specific reason for the predicted output (e.g., in step 666 ofFIG. 6B).

For instance, if the accessed feature value were to represent a value ofa numerical input feature (e.g., associated with the feature identifierof the ranked pair), FI computing system 130 may perform any of theexemplary processes described herein, in step 666, to generate elementsof textual content that include the phrase “the feature value is toohigh” when the accessed feature value exceeds the correspondingthreshold feature value (e.g., as specified within the correspondingelement of numerical parameter data), or to generate elements of textualcontent that include the phrase “the feature value is too local” whenthe accessed feature value fails to exceed the corresponding thresholdfeature value. Alternatively, if the accessed feature value were torepresent a value of a categorical input feature (e.g., associated withthe feature identifier of the ranked pair), FI computing system 130 mayperform any of the exemplary processes described herein, in step 666, togenerate elements of textual content that include the phrase “thefeature value is the threshold category” when the accessed feature valueincludes the threshold category (e.g., as specified within thecorresponding element of numerical parameter data), or to generateelements of textual content that include the phrase “the feature valueis the threshold category” when the accessed feature value fails toinclude the corresponding threshold category.

FI computing system 130 may also perform any of the exemplary processesdescribed herein to map the elements of textual content to correspondingelements of natural language, and generate an element of adverse reasondata that includes the corresponding elements of natural language (e.g.,in step 668 of FIG. 6B). As described herein, the corresponding elementsof natural language, and the element of adverse reason data, maycharacterize the association, and causal relationship, between thecorresponding feature values and the predicted output in a mannerreadily apparent to, and appreciable by, representatives and customersof the financial institution. In some instances, FI computing system 130may determine whether additional ones of the subset of ranked pairs offeature identifiers and Shapley feature values await analysis using anyof the exemplary processes described herein (e.g., in step 670 of FIG.6B). If, for example, FI computing system 130 were to determine that atleast one of the subset of ranked pairs of feature identifiers andShapley feature values await analysis (e.g., step 674; YES), exemplaryprocess 650 may pass back to step 660, and FI computing system mayperform operations that select an additional one of the subset of theranked pairs of feature identifiers and Shapley feature values.Alternatively, if FI computing system 130 were to determine noadditional ranked pairs of feature identifiers and Shapley featurevalues await analysis (e.g., step 670; NO), exemplary process 650 iscomplete in step 672.

Referring back to step 664, if FI computing system 130 were to determinethat the Shapley feature value of the selected ranked pair fails to thecorresponding threshold Shapley value, or that the accessed featurevalue fails to satisfy the predicted-positive condition (e.g., step 664;NO), FI computing system 130 may establish that the trainedShapley-splitter may be incapable of generating elements of textualcontent that establish a causal relationship between the accessedfeature value and the predicted output of the trained, gradient-boosted,decision-tree process. In some instances, FI computing system 130 mayperform any of the exemplary processes described herein to generate apartial dependency plot of numerical or categorical feature associatedwith the accessed feature value (e.g., in step 674 of FIG. 6B) and basedon an analysis of data characterizing the partial dependency plot, togenerate additional elements of textual content that establish a causalrelationship between the accessed feature value and the predicted outputof the trained, gradient-boosted, decision-tree process (e.g., in step676 of FIG. 6B).

For instance, if the accessed feature value were to represent a value ofa numerical input feature (e.g., associated with the feature identifierof the ranked pair), FI computing system 130 may perform operations instep 676, described herein, to compute a value of a Kendall rankcorrelation coefficient (e.g., a Kendall “τ) for the local partialdependency plot. Further, and as described herein, FI computing system130 may generate in step 680 additional elements of textual content thatinclude the phrase “the feature value is too high” when the computedvalue of the Kendall rank correlation coefficient exceeds a thresholdvalue, or additional elements of textual content that include the phrase“the feature value is too low” when the computed value of the Kendallrank correlation coefficient fails to exceed the threshold value. Inother instances, FI computing system 130 may perform operations, in step676, that package the accessed feature value associated with thenumerical input feature into the additional elements of textual content.Alternatively, if the accessed feature value were to represent a valueof a categorical input feature (e.g., associated with the featureidentifier of the ranked pair), FI computing system 130 may perform anyof the exemplary processes described herein to, based on an analysis ofthe data characterizing a local partial dependency plot, determine a newfeature value for the additional categorical input feature that wouldreduce a predicted numerical score (e.g., based on the application ofthe trained, gradient-boosted, decision tree process to an input datasetthat includes the new feature value), and package the new feature valueinto additional elements of textual content (e.g., also in step 676 ofFIG. 6B).

In some instances, exemplary process 650 may pass back to step 668, andFI computing system 130 may also perform any of the exemplary processesdescribed herein to map the additional elements of textual content tocorresponding elements of natural language that characterize theassociation, and causal relationship between the corresponding featurevalues and the predicted output in a manner readily apparent to, andappreciable by, representatives and customers of the financialinstitution.

Referring back to FIG. 6A, FI computing system 130 may perform any ofthe exemplary processes described herein to package the customeridentifier, the customer-specific element of predicted output data, andthe corresponding elements of adverse reason data, including theelements of natural language described herein, into correspondingportions of output data (e.g., in step 614 of FIG. 6A), and FI computingsystem 130 may perform operations that transmit the output data to acorresponding one of the additional computing systems associated withthe financial institution, which include, but are not limited to, issuersystem 301 (e.g., in step 616 of FIG. 6A). By way of example, and asdescribed herein, issuer system 301 may receive the output data from FIcomputing system 130, and may perform any of the exemplary processesdescribed herein to that parse each the elements of output data toobtain a corresponding numerical score for a corresponding customer, andbased on the corresponding numerical score, to elect to decline arequest by the corresponding customer to increase a credit limitassociated with a secured or unsecured credit-card account. In someexample, the elements of natural language may establish reasons for theadverse decision, and issuer system 301 may provision datacharacterizing the adverse decision, and the reasons, to a device of thecorresponding customer, e.g., for presentation within a digitalinterface. Exemplary process 600 is then complete in step 618.

D. Exemplary Hardware and Software Implementations

Embodiments of the subject matter and the functional operationsdescribed in this specification can be implemented in digital electroniccircuitry, in tangibly-embodied computer software or firmware, incomputer hardware, including the structures disclosed in thisspecification and their structural equivalents, or in combinations ofone or more of them. Exemplary embodiments of the subject matterdescribed in this specification, including, but not limited to,application programming interfaces (APIs) 134, 304, and 382, dataingestion engine 136, pre-processing engine 140, training engine 162,training input module 166, adaptive training and validation module 172,explainability engine 210, predictive engine 214, training engine 230,numerical feature training module 232, categorical feature trainingmodule 244, process input engine 312, post-processing engine 338, reasongeneration engine 344, selection module 346, Shapley-splitter predictivemodule 350, local PDP predictive module 366, reason generation module374, and credit modification engine 384, can be implemented as one ormore computer programs, i.e., one or more modules of computer programinstructions encoded on a tangible non transitory program carrier forexecution by, or to control the operation of, a data processingapparatus (or a computer system).

Additionally, or alternatively, the program instructions can be encodedon an artificially generated propagated signal, such as amachine-generated electrical, optical, or electromagnetic signal that isgenerated to encode information for transmission to suitable receiverapparatus for execution by a data processing apparatus. The computerstorage medium can be a machine-readable storage device, amachine-readable storage substrate, a random or serial access memorydevice, or a combination of one or more of them.

The terms “apparatus,” “device,” and “system” refer to data processinghardware and encompass all kinds of apparatus, devices, and machines forprocessing data, including, by way of example, a programmable processorsuch as a graphical processing unit (GPU) or central processing unit(CPU), a computer, or multiple processors or computers. The apparatus,device, or system can also be or further include special purpose logiccircuitry, such as an FPGA (field programmable gate array) or an ASIC(application-specific integrated circuit). The apparatus, device, orsystem can optionally include, in addition to hardware, code thatcreates an execution environment for computer programs, such as codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them.

A computer program, which may also be referred to or described as aprogram, software, a software application, a module, a software module,a script, or code, can be written in any form of programming language,including compiled or interpreted languages, or declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, or other unitsuitable for use in a computing environment. A computer program may, butneed not, correspond to a file in a file system. A program can be storedin a portion of a file that holds other programs or data, such as one ormore scripts stored in a markup language document, in a single filededicated to the program in question, or in multiple coordinated files,such as files that store one or more modules, sub-programs, or portionsof code. A computer program can be deployed to be executed on onecomputer or on multiple computers that are located at one site ordistributed across multiple sites and interconnected by a communicationnetwork.

The processes and logic flows described in this specification can beperformed by one or more programmable computers executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, such as an FPGA (field programmable gate array), an ASIC(application-specific integrated circuit), one or more processors, orany other suitable logic.

Computers suitable for the execution of a computer program include, byway of example, general or special purpose microprocessors or both, orany other kind of central processing unit. Generally, a CPU will receiveinstructions and data from a read-only memory or a random access memoryor both. The essential elements of a computer are a central processingunit for performing or executing instructions and one or more memorydevices for storing instructions and data. Generally, a computer willalso include, or be operatively coupled to receive data from or transferdata to, or both, one or more mass storage devices for storing data,such as magnetic, magneto-optical disks, or optical disks. However, acomputer need not have such devices. Moreover, a computer can beembedded in another device, such as a mobile telephone, a personaldigital assistant (PDA), a mobile audio or video player, a game console,a Global Positioning System (GPS) receiver, or a portable storagedevice, such as a universal serial bus (USB) flash drive, to name just afew.

Computer-readable media suitable for storing computer programinstructions and data include all forms of non-volatile memory, mediaand memory devices, including by way of example semiconductor memorydevices, such as EPROM, EEPROM, and flash memory devices; magneticdisks, such as internal hard disks or removable disks; magneto-opticaldisks; and CD-ROM and DVD-ROM disks. The processor and the memory can besupplemented by, or incorporated in, special purpose logic circuitry.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on a computerhaving a display unit, such as a CRT (cathode ray tube) or LCD (liquidcrystal display) monitor, for displaying information to the user and akeyboard and a pointing device, such as a mouse or a trackball, by whichthe user can provide input to the computer. Other kinds of devices canbe used to provide for interaction with a user as well; for example,feedback provided to the user can be any form of sensory feedback, suchas visual feedback, auditory feedback, or tactile feedback; and inputfrom the user can be received in any form, including acoustic, speech,or tactile input. In addition, a computer can interact with a user bysending documents to and receiving documents from a device that is usedby the user; for example, by sending web pages to a web browser on auser's device in response to requests received from the web browser.

Implementations of the subject matter described in this specificationcan be implemented in a computing system that includes a back-endcomponent, such as a data server, or that includes a middlewarecomponent, such as an application server, or that includes a front-endcomponent, such as a computer having a graphical user interface or a webbrowser through which a user can interact with an implementation of thesubject matter described in this specification, or any combination ofone or more such back-end, middleware, or front-end components. Thecomponents of the system can be interconnected by any form or medium ofdigital data communication, such as a communication network. Examples ofcommunication networks include a local area network (LAN) and a widearea network (WAN), such as the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someimplementations, a server transmits data, such as an HTML page, to auser device, such as for purposes of displaying data to and receivinguser input from a user interacting with the user device, which acts as aclient. Data generated at the user device, such as a result of the userinteraction, can be received from the user device at the server.

While this specification includes many specifics, these should not beconstrued as limitations on the scope of the invention or of what may beclaimed, but rather as descriptions of features specific to particularembodiments of the invention. Certain features that are described inthis specification in the context of separate embodiments may also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment mayalso be implemented in multiple embodiments separately or in anysuitable sub-combination. Moreover, although features may be describedabove as acting in certain combinations and even initially claimed assuch, one or more features from a claimed combination may in some casesbe excised from the combination, and the claimed combination may bedirected to a sub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems maygenerally be integrated together in a single software product orpackaged into multiple software products.

Various embodiments have been described herein with reference to theaccompanying drawings. It will, however, be evident that variousmodifications and changes may be made thereto, and additionalembodiments may be implemented, without departing from the broader scopeof the disclosed embodiments as set forth in the claims that follow.

Further, other embodiments will be apparent to those skilled in the artfrom consideration of the specification and practice of one or moreembodiments of the present disclosure. It is intended, therefore, thatthis disclosure and the examples herein be considered as exemplary only,with a true scope and spirit of the disclosed embodiments beingindicated by the following listing of exemplary claims.

What is claimed is:
 1. An apparatus, comprising: a memory storinginstructions; a communications interface; and at least one processorcoupled to the memory and the communications interface, the at least oneprocessor being configured to execute the instructions to: generate aninput dataset based on elements of first interaction data associatedwith a first temporal interval; based on an application of a trainedartificial intelligence process to the input dataset, generate outputdata representative of a predicted likelihood of an occurrence of anevent during a second temporal interval; based on an application of atrained explainability process to the input dataset, generate a firstelement of textual content that characterizes an outcome associated withthe predicted likelihood of the occurrence of the event, the firstelement of textual content being associated with a feature value of theinput dataset; and transmit a portion of the output data and the firstelement of textual content to a computing system via the communicationsinterface, the computing system being configured to generate or modifysecond interaction data based on the portion of the output data, and toprovision notification data comprising the first element of textualcontent to a device associated with the first interaction data.
 2. Theapparatus of claim 1, wherein the trained artificial intelligenceprocess comprises a trained, gradient-boosted, decision-tree process. 3.The apparatus of claim 1, wherein the feature value is associated withan input feature, and the at least one processor is further configuredto execute the instructions to: generate explainability data associatedwith the input dataset, the explainability data comprising a Shapleyfeature value associated with the input feature; obtain the featurevalue of the input feature from the input dataset, and obtain parameterdata associated with the trained explainability process and the inputfeature, the parameter data comprising a threshold feature value, athreshold Shapley value, and data characterizing a predicted-positivecondition; generate the first element of textual content based on thefeature value and portions of the explainability data and the parameterdata.
 4. The apparatus of claim 3, wherein the at least one processor isfurther configured to execute the instructions to: based on adetermination that the Shapley feature value exceeds the thresholdShapley value, and that the feature value satisfies thepredicted-positive condition, obtain a second element of textual contentassociated with the predicted-positive condition and the input feature;and perform operations that map portions of the second element oftextual content to elements of natural language associated with thefeature value, the first element of textual content comprising theelements of natural language.
 5. The apparatus of claim 4, wherein theinput feature comprises a numerical input feature, and the at least oneprocessor is further configured to execute the instructions to: based onthe determination that the Shapley feature value exceeds the thresholdShapley value, determine that (ii) the feature value exceeds thethreshold feature value or (ii) fails to exceed the threshold featurevalue; and based on the determination that the feature value exceeds, orfails to exceed, the threshold feature value, establish that the featurevalue satisfies the predicted-positive condition for the numerical inputfeature.
 6. The apparatus of claim 4, wherein, the input featurecomprises a categorical input feature, the threshold feature valuecomprises a threshold category. and the at least one processor isfurther configured to execute the instructions to: based on thedetermination that the Shapley feature value exceeds the thresholdShapley value, determine that (ii) the feature value includes thethreshold feature category or (ii) fails to include the thresholdcategory; and based on the determination that the feature valueincludes, or fails to include, the threshold category, establish thatthe feature value satisfies the predicted-positive condition for thecategorical input feature.
 7. The apparatus of claim 3, wherein the atleast one processor is further configured to execute the instructionsto: based on a determination that at least one of (i) the Shapleyfeature value fails to exceed the threshold Shapley value or (ii) thefeature value fails to satisfy the predicted-positive condition, performoperations that generate data characterizing a partial dependency plotfor the input feature; generate a second element of textual contentbased on portions of the data characterizing the partial dependencyplot; perform operations that map portions of the second element oftextual content to elements of natural language associated with thefeature value, and generate a third element of textual content based onthe elements of natural language; and transmit the portion of the outputdata and the third element of textual content to the computing systemvia the communications interface.
 8. The apparatus of claim 7, whereinthe input feature comprises a categorical input feature associated witha plurality of candidate category values. the feature value isassociated with a first one of the candidate category values, and the atleast one processor is further configured to execute the instructionsto: identify a second one of the candidate category values based on thedata characterizing the partial dependency plot; and generate the secondelement of textual content based on the second one of the candidatecategory values.
 9. The apparatus of claim 1, wherein the at least oneprocessor is further configured to execute the instructions to: obtainelements of third interaction data, each of the elements of the thirdinteraction data comprising a temporal identifier associated with atemporal interval; based on the temporal identifiers, determine that afirst subset of the elements of the third interaction data areassociated with a prior training interval, and that a second subset ofthe elements of the third interaction data are associated with a priorvalidation interval; and generate a plurality of training datasets basedcorresponding portions of the first subset, and perform operations thattrain the artificial intelligence process based on the trainingdatasets.
 10. The apparatus of claim 1, wherein the at least oneprocessor is further configured to execute the instructions to: obtain,from the memory, a plurality of validation datasets associated with thetrained, gradient-boosted, decision-tree process, and elements ofadditional output data associated with corresponding ones of thevalidation datasets, the plurality of validation datasets comprisingvalidation feature values associated with corresponding input features;and obtain explainability data associated with the validation datasets,the explainability data comprising a Shapley feature value associatedwith each of the corresponding input features; generate a plurality oftraining samples based on the validation feature values and the computedShapley feature value, each of the training samples comprising acorresponding pair of the validation feature values and the computedShapley feature value; and for at least one of the input features,perform operations that determine a threshold feature value and athreshold Shapley value based on the training samples.
 11. Acomputer-implemented method, comprising: generating, using at least oneprocessor, an input dataset based on elements of first interaction dataassociated with a first temporal interval; using the at least oneprocessor, and based on an application of a trained artificialintelligence process to the input dataset, generating output datarepresentative of a predicted likelihood of an occurrence of an eventduring a second temporal interval; using the at least one processor, andbased on an application of a trained explainability process to the inputdataset, generating a first element of textual content thatcharacterizes an outcome associated with the predicted likelihood of theoccurrence of the event, the first element of textual content beingassociated with a feature value of the input dataset; and using the atleast one processor, transmitting a portion of the output data and thefirst element of textual content to a computing system, the computingsystem being configured to generate or modify second interaction databased on the portion of the output data, and to provision notificationdata comprising the first element of textual content to a deviceassociated with the first interaction data.
 12. The computer-implementedmethod of claim 11, wherein the feature value is associated with aninput feature, and generating the first element of textual contentcomprises: generating explainability data associated with the inputdataset, the explainability data comprising a Shapley feature valueassociated with the input feature; obtaining the feature value of thenumerical input feature from the input dataset, and obtaining parameterdata associated with the trained explainability process and the inputfeature, the parameter data comprising a threshold feature value, athreshold Shapley value, and data characterizing a predicted-positivecondition for the input feature; and generating the first element oftextual content based on the feature value and portions of theexplainability data and the parameter data.
 13. The computer-implementedmethod of claim 12, wherein generating the first element of textualcontent further comprises: based on a determination that the Shapleyfeature value exceeds the threshold Shapley value, and that the featurevalue satisfies the predicted-positive condition, obtaining a secondelement of textual content associated with the predicted-positivecondition and the input feature; and performing operations that mapportions of the second element of textual content to elements of naturallanguage associated with the feature value, the first element of textualcontent comprising the elements of natural language.
 14. Thecomputer-implemented method of claim 13, wherein the input featurecomprises a numerical input feature, and the computer-implemented methodfurther comprises: based on the determination that the Shapley featurevalue exceeds the threshold Shapley value, determining, using the atleast one processor, that (ii) the feature value exceeds the thresholdfeature value or (ii) fails to exceed the threshold feature value; andbased on the determination that the feature value exceeds, or fails toexceed, the threshold feature value, determining, using the at least oneprocessor, that the feature value satisfies the predicted-positivecondition for the numerical input feature.
 15. The computer-implementedmethod of claim 13, wherein the input feature comprises a categoricalinput feature, the threshold feature value comprises a thresholdcategory, and the computer-implemented method further comprises: basedon the determination that the Shapley feature value exceeds thethreshold Shapley value, determining that (ii) the feature valueincludes the threshold feature category or (ii) fails to include thethreshold category; and based on the determination that the featurevalue includes, or fails to include, the threshold category, determiningthat the feature value satisfies the predicted-positive condition forthe categorical input feature.
 16. The computer-implemented method ofclaim 12, further comprising: based on a determination that at least oneof (i) the Shapley feature value fails to exceed the threshold Shapleyvalue or (ii) the feature value fails to satisfy the predicted-positivecondition, performing, using the at least one processor, operations thatgenerate data characterizing a partial dependency plot for the inputfeature; generating, using the at least one processor, a second elementof textual content based on portions of the data characterizing thepartial dependency plot; using the at least one processor, performingoperations that map portions of the second element of textual content toelements of natural language associated with the feature value, andgenerating a third element of textual content based on the elements ofnatural language; and transmitting, using the at least one processor,the portion of the output data and the third element of textual contentto the computing system.
 17. The computer-implemented method of claim16, wherein the input feature comprises a categorical input featureassociated with a plurality of candidate category values, the featurevalue is associated with a first one of the candidate category values,and generating the second element of textual content comprises:identifying a second one of the candidate category values based on thedata characterizing a partial dependency plot for the input feature; andgenerating the second element of textual content based on the second oneof the candidate category values.
 18. The computer-implemented method ofclaim 11, further comprising: obtaining, using the at least oneprocessor, elements of third interaction data, each of the elements ofthe third interaction data comprising a temporal identifier associatedwith a temporal interval; based on the temporal identifiers, and usingthe at least one processor, determining that a first subset of theelements of the third interaction data are associated with a priortraining interval, and that a second subset of the elements of the thirdinteraction data are associated with a prior validation interval; andusing the at least one processor, generating a plurality of trainingdatasets based corresponding portions of the first subset, andperforming operations that train the artificial intelligence processbased on the training datasets.
 19. The computer-implemented method ofclaim 11, wherein the trained artificial intelligence process comprisesa trained, gradient-boosted, decision-tree process, and thecomputer-implemented method further comprises: obtaining, using the atleast one processor, a plurality of validation datasets associated withthe trained, gradient-boosted, decision-tree process, and elements ofadditional output data associated with corresponding ones of thevalidation datasets, the plurality of validation datasets comprisingvalidation feature values associated with corresponding input features;and obtaining, using the at least one processor, explainability dataassociated with the validation datasets, the explainability datacomprising a Shapley feature value associated with each of thecorresponding input features; generating, using the at least oneprocessor, a plurality of training samples based on the validationfeature values and the computed Shapley feature value, each of thetraining samples comprising a corresponding pair of the validationfeature values and the computed Shapley feature value; and for at leastone of the input features, performing, using the at least one processor,operations that determine a threshold feature value and a thresholdShapley value based on the training samples.
 20. A tangible,non-transitory computer-readable medium storing instructions that, whenexecuted by at least one processor, cause the at least one processor toperform a method, comprising: generating an input dataset based onelements of first interaction data associated with a first temporalinterval; based on an application of a trained artificial intelligenceprocess to the input dataset, generating output data representative of apredicted likelihood of an occurrence of an event during a secondtemporal interval; and based on an application of a trainedexplainability process to the input dataset, generating an element oftextual content that characterizes an outcome associated with thepredicted likelihood of the occurrence of the event, the element oftextual content being associated with a corresponding feature value ofthe input dataset; and transmitting a portion of the output data and theelement of textual content to a computing system, the computing systembeing configured to generate or modify second interaction data based onthe portion of the output data, and to provision notification datacomprising the element of textual content to a device associated withthe first interaction data.